Re: [PATCH 2/2] mm: set PG_dma_pinned on get_user_pages*()

[John Hubbard <jhubbard@nvidia.com>]

On 07/01/2018 11:34 PM, Leon Romanovsky wrote:
> On Sun, Jul 01, 2018 at 11:10:04PM -0700, John Hubbard wrote:
>> On 07/01/2018 10:52 PM, Leon Romanovsky wrote:
>>> On Thu, Jun 28, 2018 at 11:17:43AM +0200, Jan Kara wrote:
>>>> On Wed 27-06-18 19:42:01, John Hubbard wrote:
>>>>> On 06/27/2018 10:02 AM, Jan Kara wrote:
>>>>>> On Wed 27-06-18 08:57:18, Jason Gunthorpe wrote:
>>>>>>> On Wed, Jun 27, 2018 at 02:42:55PM +0200, Jan Kara wrote:
>>>>>>>> On Wed 27-06-18 13:59:27, Michal Hocko wrote:
>>>>>>>>> On Wed 27-06-18 13:53:49, Jan Kara wrote:
>>>>>>>>>> On Wed 27-06-18 13:32:21, Michal Hocko wrote:
>>>>>>>>> [...]
>>>>> One question though: I'm still vague on the best actions to take in the
>>>>> following functions:
>>>>>
>>>>>     page_mkclean_one
>>>>>     try_to_unmap_one
>>>>>
>>>>> At the moment, they are both just doing an evil little early-out:
>>>>>
>>>>> 	if (PageDmaPinned(page))
>>>>> 		return false;
>>>>>
>>>>> ...but we talked about maybe waiting for the condition to clear, instead?
>>>>> Thoughts?
>>>>
>>>> What needs to happen in page_mkclean() depends on the caller. Most of the
>>>> callers really need to be sure the page is write-protected once
>>>> page_mkclean() returns. Those are:
>>>>
>>>>   pagecache_isize_extended()
>>>>   fb_deferred_io_work()
>>>>   clear_page_dirty_for_io() if called for data-integrity writeback - which
>>>>     is currently known only in its caller (e.g. write_cache_pages()) where
>>>>     it can be determined as wbc->sync_mode == WB_SYNC_ALL. Getting this
>>>>     information into page_mkclean() will require some plumbing and
>>>>     clear_page_dirty_for_io() has some 50 callers but it's doable.
>>>>
>>>> clear_page_dirty_for_io() for cleaning writeback (wbc->sync_mode !=
>>>> WB_SYNC_ALL) can just skip pinned pages and we probably need to do that as
>>>> otherwise memory cleaning would get stuck on pinned pages until RDMA
>>>> drivers release its pins.
>>>
>>> Sorry for naive question, but won't it create too much dirty pages
>>> so writeback will be called "non-stop" to rebalance watermarks without
>>> ability to progress?
>>>
>>
>> That is an interesting point.
>>
>> Holding off page writeback of this region does seem like it could cause
>> problems under memory pressure. Maybe adjusting the watermarks so that we
>> tell the writeback  system, "all is well, just ignore this region until
>> we're done with it" might help? Any ideas here are welcome...
> 
> AFAIR, it is per-zone, so the solution to count dirty-but-untouchable
> number of pages to take them into account for accounting can work, but
> it seems like an overkill. Can we create special ZONE for such gup
> pages, or this is impossible too?
> 

Let's see what Michal and others prefer. The zone idea intrigues me. 

thanks,
-- 
John Hubbard
NVIDIA