Add script to query for contributors between two releases by wilzbach · Pull Request #253 · dlang/tools

wilzbach · 2017-07-22T03:13:41Z

I don't know yet how we are going to integrate this on dlang.org, but this script should be more than flexible enough to accompany many possible scenarios.

Ideas:

Add to the bottom of each changelog
Separate page for each changelog (linked from changelog) (that's how the Rails people or the Rust people handle this)
All contributors page (the tool is more than fast enough, s.t. we could run this on every deploy)

dlang-bot · 2017-07-22T03:13:42Z

Thanks for your pull request, @wilzbach!

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

CyberShadow · 2017-07-23T10:50:46Z

contributors.d

+
+    File(gitComitMappingFile)
+        .byLineCopy
+        // ignore empty lines and comments, but allow `#ponce`


Yeah there's a line in the .commitmap that starts like this, but thinking about it it makes more sense to use an extra whitespace for it.

CyberShadow · 2017-07-23T10:52:17Z

I have created a small .commitmap

That sounds like something that ought to be done via the GitHub API (or just taken out of the git history).

CyberShadow · 2017-07-23T11:10:02Z

What is this .commitmap file BTW? Some existing convention? Couldn't find anything about it.

wilzbach · 2017-07-23T11:19:42Z

What is this .commitmap file BTW? Some existing convention? Couldn't find anything about it.

Yeah I will use mailmap on the next iteration.

andralex · 2017-07-23T13:21:21Z

lgtm

MartinNowak · 2017-09-01T08:31:55Z

I've already started a mailmap for dmd, https://github.com/dlang/dmd/blob/59e801d060ee13705cbf3bf73e1b4bc8a12aa60d/.mailmap.

MartinNowak

Might just be me, but this looks as if it could be done in about 10-20 lines of bash.

MartinNowak · 2017-09-01T08:33:47Z

contributors.d

+                line.skipOver(Author);
+                return line.readUser;
+            })
+            .filter!(a => a.name != "The Dlang Bot")


Maybe you'd rather want to filter out merges, --no-merges.

There's already a flag to change to this behavior. However, I wasn't sure whether we really want to exclude merges by default.

wilzbach · 2017-09-11T00:02:41Z

Might just be me, but this looks as if it could be done in about 10-20 lines of bash.

Probably, but then (1) you always run into some special behavior of bash or things that are tricky and (2) (and this is the main reason) doing it in D allows us more flexibility as I am still not sure where we will use/display this information

Yeah I will use mailmap on the next iteration.

Done. This should be good as an initial script. The .mailmap file might require some tweaking in the future, but this is something that is probably unavoidable with new users starting to contribute. At least now we use the standard git format.

wilzbach · 2017-12-04T23:04:04Z

Ping @CyberShadow @andralex @ZombineDev - can we move forward with this?
It would be awesome to finally start generating a list of all contributors for each release.

CyberShadow · 2017-12-04T23:42:35Z

contributors.d

+                return line.readUser;
+            })
+            .filter!(a => a.name != "The Dlang Bot")
+            .each!(author => authors ~= author);


Nit, unnecessary each. Instead you can do authors ~= p.stdout.....filter!(...);

CyberShadow

Looks good, seems to work as intended.

CyberShadow · 2017-12-04T23:43:36Z

contributors.d

+    import std.uni : sicmp;
+    return authors
+            .sort!((a, b) => sicmp(a.name, b.name) < 0)
+            .release


.release seems unnecessary here

CyberShadow · 2017-12-04T23:50:17Z

contributors.d

+        auto p = pipeProcess(cmd, Redirect.stdout);
+        scope(exit) enforce(wait(p.pid) == 0, "Failed to execute '%(%s %)'.".format(cmd));
+
+        static immutable Author = "Author: ";


Instead of parsing the default git log output, you can just ask it to print the data in the format you want. You probably want --pretty=format:%aN.

CyberShadow · 2017-12-04T23:55:12Z

contributors.d

+        email: ps.dropOne.front.filter!(a => a != '"').until('>').to!string
+    };
+    return author;
+}


This function is unnecessary if you use git log's format option.

wilzbach · 2017-12-05T04:40:26Z

Instead of parsing the default git log output, you can just ask it to print the data in the format you want. You probably want --pretty=format:%aN.

Well initially I did the mapping myself, before I figured that it's probably a lot better to use .mailmap.
While you have a fair point that parsing the git log is now mostly obsolete, there are still some points why I don't want to remove it:

output in the git format (--format git) or csv format (--format csv)
mapping with Gravatars (possible future)
mapping with GitHub usernames (possible future)

OTOH I don't really care that much about it. What do you think?

CyberShadow · 2017-12-05T07:31:19Z

Sorry, I think there's a misunderstanding. I meant, you're expending a non-trivial amount of code/logic for parsing the default git log output format. Whereas, we could use the --pretty=format:... switch to git log to ask it to give us only the data we want in the format that's easiest for us to parse.

wilzbach · 2017-12-08T11:31:20Z

Somehow my message got lost due to the great WiFi on German trains....
Anyhow:

Whereas, we could use the --pretty=format:... switch to git log to ask it to give us only the data we want in the format that's easiest for us to parse.

"+3 −16" - can't argue much about this!

-> Will merge this now & look into integrating it with dlang.org

MartinNowak · 2017-12-16T19:53:21Z

.mailmap

+Walter Waldron <WalterWaldron@users.noreply.github.com>
+Yazan Dabain <yazan.dabain@arabiaweather.com> <yazan.dabain@gmail.com>
+Михаил Страшун <public@dicebot.lv> <m.strashun@gmail.com>
+Yao Gómez <yao.gomez@gmail.com> <yaoltzin@gmail.com>


Should we remove the dlang repo specific mailmaps then?

dlang/dmd#7452

MartinNowak · 2017-12-16T19:54:02Z

contributors.d

+            enforce(wait(p.pid) == 0, "Failed to execute '%(%s %)'.".format(cmd));
+        }
+
+        auto cmd = ["git", "-c", "mailmap.file=%s".format(config.mailmapFile), "-C", repo, "log", "--use-mailmap", "--pretty=format:%aN|%aE"];


git -C only works with newer versions of git

git 1.8.3 is an ancient dinosaur (more than four years old) 😉

std.process functions allow you to specify a working directory anyway.

CyberShadow reviewed Jul 23, 2017

View reviewed changes

MartinNowak reviewed Sep 1, 2017

View reviewed changes

wilzbach force-pushed the contributors branch 3 times, most recently from d863be3 to c301e82 Compare September 11, 2017 00:01

wilzbach mentioned this pull request Sep 11, 2017

Add a simple overview of all contributors dlang/dlang.org#1893

Merged

Add script to query for contributors between two releases

e3dbef2

wilzbach force-pushed the contributors branch from c301e82 to e3dbef2 Compare December 4, 2017 23:02

CyberShadow reviewed Dec 4, 2017

View reviewed changes

CyberShadow approved these changes Dec 4, 2017

View reviewed changes

CyberShadow reviewed Dec 4, 2017

View reviewed changes

Address review

b6ecfa7

Use git log --pretty=format

6bb4c24

wilzbach merged commit 404256f into dlang:master Dec 8, 2017

wilzbach deleted the contributors branch December 8, 2017 11:31

wilzbach mentioned this pull request Dec 9, 2017

Show all contributors of the upcoming release dlang/dlang.org#1949

Closed

MartinNowak reviewed Dec 16, 2017

View reviewed changes

wilzbach mentioned this pull request Dec 16, 2017

Removed repo-specific mailmap in favor of dlang/tools/.mailmap dlang/dmd#7452

Merged

wilzbach mentioned this pull request Dec 21, 2017

check-in generated contributors list dlang/dlang.org#2015

Merged

Uh oh!

Conversation

wilzbach commented Jul 22, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dlang-bot commented Jul 22, 2017

Bugzilla references

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

CyberShadow commented Jul 23, 2017

Uh oh!

CyberShadow commented Jul 23, 2017

Uh oh!

wilzbach commented Jul 23, 2017

Uh oh!

andralex commented Jul 23, 2017

Uh oh!

MartinNowak commented Sep 1, 2017

Uh oh!

MartinNowak left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wilzbach commented Sep 11, 2017

Uh oh!

wilzbach commented Dec 4, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

CyberShadow left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wilzbach commented Dec 5, 2017

Uh oh!

CyberShadow commented Dec 5, 2017

Uh oh!

wilzbach commented Dec 8, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

wilzbach commented Jul 22, 2017 •

edited

Loading