From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46008C43460 for ; Tue, 11 May 2021 18:25:24 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B1E2E61919 for ; Tue, 11 May 2021 18:25:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B1E2E61919 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=pernos.co Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CBC896B006E; Tue, 11 May 2021 14:25:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C6CCD6B0070; Tue, 11 May 2021 14:25:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AE6296B0072; Tue, 11 May 2021 14:25:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0038.hostedemail.com [216.40.44.38]) by kanga.kvack.org (Postfix) with ESMTP id 91E176B006E for ; Tue, 11 May 2021 14:25:22 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 4E636BBF6 for ; Tue, 11 May 2021 18:25:22 +0000 (UTC) X-FDA: 78129777684.19.0CC4410 Received: from mail-lf1-f44.google.com (mail-lf1-f44.google.com [209.85.167.44]) by imf23.hostedemail.com (Postfix) with ESMTP id B499CA000187 for ; Tue, 11 May 2021 18:25:11 +0000 (UTC) Received: by mail-lf1-f44.google.com with SMTP id h4so30080077lfv.0 for ; Tue, 11 May 2021 11:25:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pernos.co; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=J9RyWF22KzwLKArKaPRg8pNWC1t+VofvpkH4MjpQva4=; b=aszYMCricCo5Ag2eeDk8OCI5dD4USw7RzHUgzv2ez4K73y68yotiOfwA5n075q4HxL o/D6aSm6ed5gNiXxemb/fl3o9pA7tsWjzQWfvmnAxCq7CP0zulbRGJCe/id7PNoi7xY2 EJnBGhHxuido7XftseRe1uF8HKYP60atm4Rhg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=J9RyWF22KzwLKArKaPRg8pNWC1t+VofvpkH4MjpQva4=; b=m4e3QoJr02CygbGoLxivN0ggAtEDjHYi+lErEKG4ywS3+2aGUaxmDHehIwR+4UGHUv VvyhCIAyyiQ2CGxYxf/5l4gTPDsRCsvzdLy+EFa9HAOtaujMSgGi9ycEXgXjDxhCqZme LcvNedQoFtvlcRgQZ4e+SGLEur6GH0hG8o9uGN6CgiNbeZIs3qI0nMjRLMzAUBUIvlfa Syx5lIux8BJe8UUAwvhDScXp30wvnTrDX3UMEV5FTCAP5kEqDlZXp7nVyKP6ueB9KeZh gJv7pjsA/hUh8vfXywHg/oFS1+Dgz9OT5trg4PfzBXCMIjd5Vm+UuNlxKG4C5v0p8Zka Kw9g== X-Gm-Message-State: AOAM531DTYPHvMyUOeJVgJEoOI/u+0EOPHE3vmUrKeAWStcyzKH+H5/D BQ8hjjpy8eXv/NhAOOwoZhNPG3/pVQa5vZqpa0s3Nw== X-Google-Smtp-Source: ABdhPJwgY7J4Ug37iFaEo37AHTgndWxjBuFbh4z5GujTsg43tHgFCKVtGnYy+QoX5ltXg2X46reWPWHpq6VaTI+s0Vw= X-Received: by 2002:a05:6512:3aa:: with SMTP id v10mr21501747lfp.263.1620757520265; Tue, 11 May 2021 11:25:20 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Kyle Huey Date: Tue, 11 May 2021 11:25:07 -0700 Message-ID: Subject: Re: Userspace notifications for observing userfaultfd faults To: Axel Rasmussen Cc: "Robert O'Callahan" , Andrea Arcangeli , Andrew Morton , Hugh Dickins , Peter Xu , Ingo Molnar , Linux-MM , LKML Content-Type: text/plain; charset="UTF-8" Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=pernos.co header.s=google header.b=aszYMCri; spf=none (imf23.hostedemail.com: domain of khuey@pernos.co has no SPF policy when checking 209.85.167.44) smtp.mailfrom=khuey@pernos.co; dmarc=pass (policy=quarantine) header.from=pernos.co X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: B499CA000187 X-Stat-Signature: 98hb1ozh4hz1obxxupy17jaefk15o4sa Received-SPF: none (pernos.co>: No applicable sender policy available) receiver=imf23; identity=mailfrom; envelope-from=""; helo=mail-lf1-f44.google.com; client-ip=209.85.167.44 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1620757511-55242 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, May 11, 2021 at 11:12 AM Axel Rasmussen wrote: > > On Mon, May 10, 2021 at 5:38 PM Robert O'Callahan wrote: > > > > For rr (https://rr-project.org) to support recording and replaying > > applications that use userfaultfd, we need to observe that a task we > > are controlling has blocked on a userfault. Currently this is very > > difficult to do, especially if a task blocks on a userfault on a page > > where some other task has already triggered a userfault, so no new > > userfaultfd event is generated. We also need to observe which page has > > been faulted on so we can determine when the fault has been serviced > > and the task is ready to run again. > > > > I've tried to find workarounds with existing APIs and it doesn't seem > > tractable. See https://github.com/rr-debugger/rr/issues/2852#issuecomment-837514946 > > for some thoughts about that. > > > > It seems to me that a sufficient API for us would be a new software > > perf event, e.g. PERF_COUNT_SW_USERFAULTS, with an associated > > PERF_SAMPLE_ADDR that would give us the address of the page. Does that > > sounds like a reasonable thing to add? > > Is some combination of bpf and kprobes a possible solution? There are > some seemingly relevant examples here: > https://github.com/iovisor/bpftrace/blob/master/docs/tutorial_one_liners.md > > I haven't tried it, but it seems like attaching to handle_userfault() > would give similar information to perf_count_sw_page_faults, but for > userfaults. My understanding is that using bpf/kprobes requires new permissions that are both not currently required by rr and would not be required by our proposed solution. - Kyle