Skip to content
This repository was archived by the owner on Feb 25, 2025. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
de67def
script for fetching correct flutter version
Feb 26, 2020
aaf5c00
change cirrus yml for the script location
Feb 26, 2020
ab4396b
change location of the script. Add it to tools
Feb 26, 2020
fd54619
cirrus still does not see the script. repeat path change from the pre…
Feb 26, 2020
bc4bc65
Looks like script worked correctly. do not change directory.
Feb 26, 2020
66595f0
change directory back to build root after scriot is run
Feb 26, 2020
2dd118c
script runs ok. Still not able to find the bin directory. go all the …
Feb 26, 2020
5c49aa5
still can't see the bin directory. carry the script content to cirrus…
Feb 26, 2020
e60e862
get the last commit id of the right repository
Feb 26, 2020
7a1325f
content of the script worked in cirrus. call the script from tools
Feb 26, 2020
6d3afaa
cannot find the script under tools
Feb 26, 2020
63c3a8a
print the current path in the script to see why cirrus cannot see bin…
Feb 26, 2020
939a9c0
move to flutter path before running update packages
Feb 26, 2020
daa8884
tests run now. remove print outs
Feb 26, 2020
5887ede
error if the ENGINE_PATH is not set. exit the script
Feb 27, 2020
4c9fce0
addressing reviewer comments
Feb 28, 2020
e88bcb2
engine branch name on cirrus logs doesn't make sense
Feb 28, 2020
4712ab5
fix typo
Feb 28, 2020
1fc87e9
change the directory of branch calculation
Feb 28, 2020
618a176
remove extra logs
Feb 28, 2020
c39a4ae
addressing PR comments. Testing CIRRUS_CI env variable
Feb 28, 2020
156bd65
adding CIRRUS_CI check
Feb 28, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions .cirrus.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,9 @@ web_shard_template: &WEB_SHARD_TEMPLATE
./flutter/tools/gn --unoptimized --full-dart-sdk
ninja -C out/host_debug_unopt
fetch_framework_script: |
mkdir -p $FRAMEWORK_PATH
cd $FRAMEWORK_PATH
git clone https://github.com/flutter/flutter.git
cd flutter
cd $ENGINE_PATH/src/flutter/tools
./clone_flutter.sh
cd $FRAMEWORK_PATH/flutter
bin/flutter update-packages --local-engine=host_debug_unopt
script:
- dart --enable-asserts $FRAMEWORK_PATH/flutter/dev/bots/test.dart --local-engine=host_debug_unopt
Expand Down
64 changes: 64 additions & 0 deletions tools/clone_flutter.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
#!/bin/bash
set -e
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd also add set -x so all the commands we run here are logged

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the suggestion!

set -x

if [[ "$CIRRUS_CI" = false || -z $CIRRUS_CI ]]
then
echo "This script is aimed to be run on CI environments. Do not run locally."
exit 1
fi

if [[ -z $ENGINE_PATH ]]
then
echo "Engine path should be set to run the script."
exit 1
fi

# Go to the engine git repo to get the date of the latest commit.
cd $ENGINE_PATH/src/flutter

# Special handling of release branches.
ENGINE_BRANCH_NAME=`git branch | grep '*' | cut -d ' ' -f2`
versionregex="^v[[:digit:]]+\."
ON_RELEASE_BRANCH=false
echo "Engine on branch $ENGINE_BRANCH_NAME"
if [[ $ENGINE_BRANCH_NAME =~ $versionregex ]]
then
echo "release branch $ENGINE_BRANCH_NAME"
ON_RELEASE_BRANCH=true
fi

# Get latest commit's time for the engine repo.
# Use date based on local time otherwise timezones might get mixed.
LATEST_COMMIT_TIME_ENGINE=`git log -1 --date=local --format="%ad"`
echo "Latest commit time on engine found as $LATEST_COMMIT_TIME_ENGINE"

# Do rest of the task in the root directory
cd ~
mkdir -p $FRAMEWORK_PATH
cd $FRAMEWORK_PATH
# Clone the Flutter Framework.
git clone https://github.com/flutter/flutter.git
cd flutter

FRAMEWORK_BRANCH_NAME=`git branch | grep '*' | cut -d ' ' -f2`
if [[ "$ON_RELEASE_BRANCH" = true && ENGINE_BRANCH_NAME != FRAMEWORK_BRANCH_NAME ]]
then
echo "For a release framework and engine should be on the same version."
echo "Switching branches on Framework to from $FRAMEWORK_BRANCH_NAME to $ENGINE_BRANCH_NAME"
# Switch to the same version branch with the engine.
# If same version branch does not exits, fail.
SWITCH_RESULT=`git checkout $ENGINE_BRANCH_NAME` || true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wont || true make this always true?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. If we don't put it, when there is an error (such as branch not found) the script automatically ends without printing out the logs I added.

When we use this if the branch exists the SWITCH_RESULT is a non empty string if it fails it is an empty string

if [[ -z "$SWITCH_RESULT" ]]
then
echo "$ENGINE_BRANCH_NAME Branch not found on framework. Quit."
exit 1
fi
fi

# Get the time of the youngest commit older than engine commit.
# Git log uses commit date not the author date.
# Before makes the comparison considering the timezone as well.
COMMIT_NO=`git log --before="$LATEST_COMMIT_TIME_ENGINE" -n 1 | grep commit | cut -d ' ' -f2`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if the latest engine commit is after the latest framework commit? Will this say there's no commit to reset to? If it leaves the framework at ToT, then multiple runs of this script at the same engine commit could still yield different framework versions...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is deterministically getting the youngest framework commit older than the engine commit. For example engine commit is 10pm PST. The framework tree was red for half a day. The framework commit will be 10am PST.

Yes you are right. Different commits running at the same time on cirrus, since their head is different, will be fetching a different version of the framework repo.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah cool - at first I read it as "oldest commit younger than the engine commit".

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So yes, next question is about branches. How does this behave if you're running on a release branch in the engine? We may want to have this detect if you're not on master, and then fetch the branch by the same name in the framework (and fail if there's no branch by the same name in the framework).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we worry about https://stackoverflow.com/questions/12934044/git-parent-commit-is-younger-than-descendant? Perhaps topological age should take precedence over actual date?

@tvolkert How do we run tests on CI for branches? Also, how do we keep LUCI recipes stable? A change in LUCI recipe can break our tests in older commits too (in fact it did just last week).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On flutter/engine there isn't any branches for 'dev', 'beta', 'stable'. Is the suggestion for managing the hotfixes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yjbanov On stackoverflow article:

  1. The commit date can be changed that is true. On the other hand we are simply trying to prevent both trees being closed on an error on flutter/flutter. We can add this to the Guidelines and simply ask the engine contributors not to change the date.
  2. git log --before and --after comments are always using commit date, so all the other parts on the stackoverflow discussion regarding the rebases are irrelevant to line 21 of the script.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yjbanov infra team is working on the ability to pin the recipe version to the engine.

@nturgut these won't be dev, beta, or stable - they'll be release branches (one per dev release, created on both the framework and the engine).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Hixie @tvolkert I added the checks for the branch. @yjbanov I added a comment for your concern.

Even though this script works locally, I am not sure how it will run for the release process with cirrus CI. as far I as understand cirrus only runs on detached head. 'master' branches is used for post submit benchmarks. Once the cloning is done, there isn't any other configuration for a specific branch.

In my opinion we should either use LUCI for the release process or change the clone script of cirrus https://cirrus-ci.org/guide/tips-and-tricks/#custom-clone-command

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the github history I couldn't find any instances when a child commit date is older than the parent's, so it looks like Github's "squash and merge" button does the right thing for us. This can only happen if someone pushes using the command line, and they would really have to go out of their way to change the dates. I think we're good.

Recipe pinning SGTM. Or better, move them into the respective repos.

echo "Using the flutter/flutter commit $COMMIT_NO";
git reset --hard $COMMIT_NO