Selective lei filters and replies to old threads

Selective lei filters and replies to old threads

[Toke Høiland-Jørgensen]

Hi everyone

I've been happily using lei in place of mailing list subscriptions for a
while now, but ran into an issue recently while trying to limit the
volume of email, which I am hoping someone has a good solution for:

Basically, what I was trying to do was to avoid following the firehose
of patches that is netdev@vger, and instead selectively pull in patches
for the areas I care about (which mostly boils down to avoiding all the
driver-only patches). To achieve this, I've set up a lei search using
the handy dfn: prefix for searches. In simplified form I've gone from a
search like:

q = l:netdev.vger.kernel.org AND rt:1.month.ago..

to

q = dfn:net/core/* AND rt:1.month.ago...

(my full query has a few more terms but this is enough to illustrate the
problem).

Now, the above query works beautifully to bring down the volume of
patches to a more manageable level (phew), but I have run into a problem
with follow-ups, which I'll try to describe as I understand it:

I'm using the --threads parameter to 'lei q' which nicely pulls down the
whole thread whenever one patch in the series matches[0]. However, if a
reply shows up later, which does not in itself match the dfn: query, but
is part of a thread that does, this reply will only be downloaded if the
original thread still matches[1]. Which it doesn't necessarily do
anymore because of the time limit parameter added by lei which only
pulls in new emails since the last one downloaded.

I can get around the default lei time limit by adding
--remote-fudge-time=1.month to 'lei up', but that makes lei pull in
thousands of mails on every update, which seems a bit wasteful. And I'll
still miss replies to threads that are older than the 1-month time
limit.

So really I think what I'm looking for is a way to construct a query
that uses the send time of the individual (reply) messages, but applies
the dfn: match to the whole thread. Is there a way to do that? And/or
does anyone have any other neat tricks they are using to construct a
similar selective mail feed without missing stuff? :)

-Toke



[0] Side note: if anyone has a nice trick to specifically exclude Greg's
stable patch bombs from this, I would be interested in hearing about
that as well :)

[1] An example: I was missing this reply in my local mailbox even though
the original thread was there:
https://lore.kernel.org/all/64dcf5834c4c8_23f1f8294fa@willemb.c.googlers.com.notmuch/
Adding the --remote-fudge-time parameter pulled it in.

Re: Selective lei filters and replies to old threads

[Greg KH]

On Thu, Aug 17, 2023 at 04:15:54PM +0200, Toke Høiland-Jørgensen wrote:
> [0] Side note: if anyone has a nice trick to specifically exclude Greg's
> stable patch bombs from this, I would be interested in hearing about
> that as well :)

I thought those stable patch bombs had a X- header on them, like:
	X-stable: review
and
	X-Patchwork-Hint: ignore
but somehow it looks like I'm only adding that to the first message in
the stable-patch-bomb series.

Would it help for me to fix my side and add those headers to make it
easier for you to be able to ignore them?

thanks,

greg k-h

Re: Selective lei filters and replies to old threads

[Toke Høiland-Jørgensen]

Greg KH <gregkh@linuxfoundation.org> writes:

> On Thu, Aug 17, 2023 at 04:15:54PM +0200, Toke Høiland-Jørgensen wrote:
>> [0] Side note: if anyone has a nice trick to specifically exclude Greg's
>> stable patch bombs from this, I would be interested in hearing about
>> that as well :)
>
> I thought those stable patch bombs had a X- header on them, like:
> 	X-stable: review
> and
> 	X-Patchwork-Hint: ignore
> but somehow it looks like I'm only adding that to the first message in
> the stable-patch-bomb series.
>
> Would it help for me to fix my side and add those headers to make it
> easier for you to be able to ignore them?

I don't think it's possible to match on arbitrary headers in Lore
searches (maybe public-inbox could be taught that?), but it would
certainly aid in adding local filtering after fetching which would work
for me as a workaround. So yes, please! :)

-Toke

Re: Selective lei filters and replies to old threads

[Greg KH]

On Fri, Aug 18, 2023 at 08:02:34PM +0200, Toke Høiland-Jørgensen wrote:
> Greg KH <gregkh@linuxfoundation.org> writes:
> 
> > On Thu, Aug 17, 2023 at 04:15:54PM +0200, Toke Høiland-Jørgensen wrote:
> >> [0] Side note: if anyone has a nice trick to specifically exclude Greg's
> >> stable patch bombs from this, I would be interested in hearing about
> >> that as well :)
> >
> > I thought those stable patch bombs had a X- header on them, like:
> > 	X-stable: review
> > and
> > 	X-Patchwork-Hint: ignore
> > but somehow it looks like I'm only adding that to the first message in
> > the stable-patch-bomb series.
> >
> > Would it help for me to fix my side and add those headers to make it
> > easier for you to be able to ignore them?
> 
> I don't think it's possible to match on arbitrary headers in Lore
> searches (maybe public-inbox could be taught that?), but it would
> certainly aid in adding local filtering after fetching which would work
> for me as a workaround. So yes, please! :)

Turned out my scripts were broken, and the header I was supposed to be
putting on all stable patches was no longer happening (something changed
in git format-patch's output I think.)

So this should be now fixed up with the latest 5.10.192-rc1 announcement
and emails.

thanks,

greg k-h

Re: Selective lei filters and replies to old threads

[Toke Høiland-Jørgensen]

Greg KH <gregkh@linuxfoundation.org> writes:

> On Fri, Aug 18, 2023 at 08:02:34PM +0200, Toke Høiland-Jørgensen wrote:
>> Greg KH <gregkh@linuxfoundation.org> writes:
>> 
>> > On Thu, Aug 17, 2023 at 04:15:54PM +0200, Toke Høiland-Jørgensen wrote:
>> >> [0] Side note: if anyone has a nice trick to specifically exclude Greg's
>> >> stable patch bombs from this, I would be interested in hearing about
>> >> that as well :)
>> >
>> > I thought those stable patch bombs had a X- header on them, like:
>> > 	X-stable: review
>> > and
>> > 	X-Patchwork-Hint: ignore
>> > but somehow it looks like I'm only adding that to the first message in
>> > the stable-patch-bomb series.
>> >
>> > Would it help for me to fix my side and add those headers to make it
>> > easier for you to be able to ignore them?
>> 
>> I don't think it's possible to match on arbitrary headers in Lore
>> searches (maybe public-inbox could be taught that?), but it would
>> certainly aid in adding local filtering after fetching which would work
>> for me as a workaround. So yes, please! :)
>
> Turned out my scripts were broken, and the header I was supposed to be
> putting on all stable patches was no longer happening (something changed
> in git format-patch's output I think.)
>
> So this should be now fixed up with the latest 5.10.192-rc1 announcement
> and emails.

Yup, I see the tags - great, I'll set up some filters. Thanks! :)

-Toke