Option to ignore forks

[heaths]

When searching for code, it would be preferable to see authoritative results only - to ignore forks which can also cause needles scrolling through pages of results.

For example, this search shows a lot of forks. They might help, but may also be far behind the main repo from which they were forked.

GitHub already knows if a repo is a fork so I'm hoping that can permeate to search results and we can exclude them. Or maybe even exclude them by default and let searchers opt into seeing them. I imagine there's good reasons to search forks, but doesn't seem helpful by default.

[bagder]

Plain forks would be easy for GitHub to ignore and it would be most useful.

I do however have an extended problem, and that is that I author a library (curl/curl) that hundreds of other projects import into their repositories. If I try to find users of this library, like for example if I want to check if I can find applications that use a particular option, there are literally thousands of forks and code imports that make it next to impossible to find what I'm looking for.

Test case: Look for CURLOPT_HEADERFUNCTION

[look]

Here's that same query with the new code search.

What kind of results or filters are you looking for to meet your use case? (I'm not familiar with the code in the results so it's hard for me to judge the quality.)

[jsoref]

Here's an example:
https://cs.github.com/?scopeName=All+repos&scope=&q=%22curl_thread_t+Curl_thread_create%28unsigned+int+%28*func%29+%28void+*%29%2C+void+*arg%29%22

Consider these hits:

nrfconnect/sdk-nrf
‎ext/curl/lib/curl_threads.c‎
1 match
C
main

  return 0;
}

curl_thread_t Curl_thread_create(unsigned int (*func) (void *), void *arg)
{
  curl_thread_t t = malloc(sizeof(pthread_t));
  struct Curl_actual_call *ac = malloc(sizeof(struct Curl_actual_call));

https://github.com/nrfconnect/sdk-nrf/blob/b6f76303347d8d5cf37c4186b4738cbce41b20c1/ext/curl/lib/curl_threads.c

It's a copy of curl's -- not sure the specific version, and I'm not going to try.

whitecatboard/Lua-RTOS-ESP32
‎components/curl/lib/curl_threads.c‎
1 match
C
master

  return 0;
}

curl_thread_t Curl_thread_create(unsigned int (*func) (void *), void *arg)
{
  curl_thread_t t = malloc(sizeof(pthread_t));
  struct curl_actual_call *ac = malloc(sizeof(struct curl_actual_call));

Ditto.

The point is that one doesn't want to see a committed copy of the code, just consumer code (which will be in the same repo, since, after all, they vendored the code, so presumably, they're making some calls to the library).

[jsoref]

But, even given your example, or rather, a variant

Here's one where someone uploaded a copy of curl but didn't tell github it was a fork of curl/curl:

MikeRalphson/curl
‎lib/url.c‎
1 match
C
master

    if(!data->set.err)
      data->set.err = stderr;
    break;
  case CURLOPT_HEADERFUNCTION:
    /*
     * Set header write callback
     */

https://github.com/MikeRalphson/curl/blob/cba16bf38dc0fe40d65b5f4e266fbf96de014637/lib/url.c

[look]

@heaths Thanks for the feedback. This is on our roadmap, so stay tuned!

[kidonng]

I don't know about the technical details or how many forks are out there so I may be wrong, but implementing fork filtering should help scaling as you can put off the indexing work for forks.

@Dentrax Ignoring forks is definitely going to be the default, it's just not implemented. Also, better use the existing syntax for regular searches fork:true / fork:only.

[nickserv]

For reference, Sourcegraph automatic alt ignores archives and forks from GitHub by default. Both can be included with Boolean flags.

[Dentrax]

Kindly ping @look. I just wanted to circle back on this ask.

[look]

We still plan to do this, but have been focusing on building out the system before adding additional filters to the index.

[nicowilliams]

This is the single biggest problem with GitHub search by far. There is so so much noise from forks in any search for any common thing. Want to know who uses some OpenSSL function? No, you can't get the answer, because you'll see the thousands of forks of OpenSSL first, with occasional hits mixed in, so you can't even skip to page 500 or something to begin finding non-fork hits. Basically, GitHub search is useless to me. Like right now I'm looking for "tpm2" in code written in Rust, and as esoteric as one might think that search might be... nope, the search results are full of noise from forks.

[Trinhan87]

When searching for code, it would be preferable to see authoritative results only - to ignore forks which can also cause needles scrolling through pages of results.

For example, this search shows a lot of forks. They might help, but may also be far behind the main repo from which they were forked.

GitHub already knows if a repo is a fork so I'm hoping that can permeate to search results and we can exclude them. Or maybe even exclude them by default and let searchers opt into seeing them. I imagine there's good reasons to search forks, but doesn't seem helpful by default.

[look]

Thanks for your patience, everyone. There is now an option to exclude forks (or limit your search to forks).

To exclude forks from your results:

NOT is:fork

To search only fork repositories:

is:fork

For more information, please see the documentation.

[jsoref]

Thanks.

It'd be nice if non owners of repositories could crowdsource a tag for unregistered-fork as exemplified in https://github.com/orgs/community/discussions/13126#discussioncomment-2909457.

As a given repository intentionally bypassed the fork button, we can't rely on the owners to add a tag...

Do you have any suggestions for how to address this edge / plans for it going forward?

[look]

For exact duplicate files, our search engine should already pick the "best" repository to show, which should be the original (or a more highly ranked fork in the case when the original repository has been superseded).

That doesn't cover files that have changed, either in the original repository or the fork. For these, we are looking into adding document similarity and improving our ranking.

In the meantime, I recommend a save scope populated with repositories you want to exclude.

[monperrus]

I confirm that NOT is:fork works as expected

https://github.com/search?q=%22Decompiler+and+security+analysis+tool+for+all+deployed+contracts%22+NOT+is%3Afork&type=code

[Func86]

The advanced search page can't generate the correct syntax for code search, it uses fork:true and fork:only, which are for repo search.

Possible unrecognized qualifier, searching for this term literally