diff options
| author | Peter Xu | 2022-05-12 20:22:54 -0700 |
|---|---|---|
| committer | Andrew Morton | 2022-05-13 07:20:10 -0700 |
| commit | 166f3ecc0daf0c164bd7e2f780dbcd1e213ac95f (patch) | |
| tree | bf8bad63911bb15b471d76bbc32e7fa1a3d5177e /mm/hugetlb.c | |
| parent | 229f3fa778c50edf12b34bd91b2025645a11bb94 (diff) | |
mm/hugetlb: hook page faults for uffd write protection
Hook up hugetlbfs_fault() with the capability to handle userfaultfd-wp
faults.
We do this slightly earlier than hugetlb_cow() so that we can avoid taking
some extra locks that we definitely don't need.
Link: https://lkml.kernel.org/r/20220405014901.14590-1-peterx@redhat.com
Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Mike Kravetz <mike.kravetz@oracle.com>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Jerome Glisse <jglisse@redhat.com>
Cc: "Kirill A . Shutemov" <kirill@shutemov.name>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
Cc: Nadav Amit <nadav.amit@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Diffstat (limited to 'mm/hugetlb.c')
| -rw-r--r-- | mm/hugetlb.c | 20 |
1 files changed, 20 insertions, 0 deletions
diff --git a/mm/hugetlb.c b/mm/hugetlb.c index fc31a8f49067..0a2b3b8e765b 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5737,6 +5737,26 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma, if (unlikely(!pte_same(entry, huge_ptep_get(ptep)))) goto out_ptl; + /* Handle userfault-wp first, before trying to lock more pages */ + if (userfaultfd_wp(vma) && huge_pte_uffd_wp(huge_ptep_get(ptep)) && + (flags & FAULT_FLAG_WRITE) && !huge_pte_write(entry)) { + struct vm_fault vmf = { + .vma = vma, + .address = haddr, + .real_address = address, + .flags = flags, + }; + + spin_unlock(ptl); + if (pagecache_page) { + unlock_page(pagecache_page); + put_page(pagecache_page); + } + mutex_unlock(&hugetlb_fault_mutex_table[hash]); + i_mmap_unlock_read(mapping); + return handle_userfault(&vmf, VM_UFFD_WP); + } + /* * hugetlb_wp() requires page locks of pte_page(entry) and * pagecache_page, so here we need take the former one |