create TM2 test example for 3 zones and TVPB

[bstabler]

• @lmz is working on data prep
• @bstabler is working on expressions
• @toliwaga is working on software

[lmz]

See also: https://github.com/ActivitySim/activitysim/wiki/MTC-TVPB-Test-Data

I'm converting skims to OMX and posting on Box (@bstabler and @toliwaga already have edit access)

For next steps, how do I setup the test run to use these files?

[bstabler]

Thanks @lmz. It looks like the data inputs are coming along. Can you also post your scripts for converting the files since we may need to make some updates once we start testing these. For now, I'll setup the test environment since I need to develop the expression files to go with these inputs. Once I get it basically working, which may require a bit of back and forth between us, I'll post it for you to try. How does that sound?

[lmz]

Sure, any preference where they ought to go? I can put them in travel-model-two or in this repo.

[bstabler]

Or maybe the box account folder for now? Any of the locations is fine....if this repo then create a new folder under other_resources. Thanks.

[lmz]

To continue on from our conversation today - I need some clarity about what actually needs to be done for this (which is why I thought it would be easier to understand if something is runnable to see what fails). For example:

• I would think we would just use householdData_3.csv as the household input file, personData_3.csv for the person input file. It's not clear to me what changes are required to those. Zone renumbering?

• Also, do input land use data files need to be combined? Can the test run be configured to use multiple files?

• Assuming the tour files are needed for the tour destinations and times, do the individual and joint tour files need to be appended? I don't know what other changes would be necessary. Zone renumbering?

• The maz-based skims are in text form and it seems like they're already the right format (but without headers -- which I find unacceptable...!)

[bstabler]

Sure. Here's what I think still needs to be done:

• for the synthetic households and persons, we need to make sure all the fields required by the tour mode choice UEC are in the input files. For example, expression valueOfTime refers to person.value_of_time.
• for the land use data, we need to make sure all the fields required by the tour mode choice UEC are in the input file and all the land use data is in one file. I think this means merging maz_data.csv, accessibilities.csv, and mgraParkingCost.csv.
• for the tour data, we need to merge indivTourData_3.csv and jointTourData_3.csv into one tour table and also create the joint_tour_participants table. We also need to make sure all the fields required by the tour mode choice UEC are in the input files.
• for the maz-based skims, we need to add headers

In terms of formats, see the examples referenced at https://github.com/ActivitySim/activitysim/wiki/MTC-TVPB-Test-Data. Thanks.

[lmz]

When running this example, what's the mechanism used to read the tour files (instead of generating them with the other submodels)? I would think it would be something like a submodel (or more than one) that reads these files rather than doing the cdap / school_location / workplace_location , etc. As such, I think it would make sense for that submodel to read the files as is and do some limited processing, consistent with the original submodel(s). I would think that would need to do processing anyway, so why not make it read the raw tm2 files as they are and do all the work, rather than having some pre-processing scripting and some work in there?

Where is the settings.yml / setup so far for a 3 zone mtc/tam example ?

[bstabler]

For now, the mechanism to read the tour file will be to restart the pipeline at tour mode choice. To do so, we'll create a pipeline file of pandas DataFrames from the TM2 CSV files. Longer term, @toliwaga and I discussed creating an initialize_tours submodel to read this table from disk (like initialize_households). I like the idea of putting the pre-processing logic into an initialize_tours expression file, but we don't have that functionality currently, so I was thinking we'd write a script. Once we have the script, then we could convert the logic into an expressions file.

Here is the current example 3 zone setup. I'm working on building a TM2 example setup based on the data you're posting.

[lmz]

Ok, forgive me for my dumb questions (I'm still an ActivitySim noob), but I don't understand what triggers the model to "restart the pipeline at tour mode choice". What command would I run? Where is this reflected in a settings.yaml? This one doesn't include the models section

[bstabler]

@lmz - I'm concerned we're missing each other a bit on this. I'm thinking you provide the reasonably well formatted data inputs and then I'll build and test the example (and possibly make revisions to the inputs if needed (and/or in coordination with you)). We currently don't have out-of-the-box functionality for this exact use case so I'm going to have to put something together for now. Once I get the example working, I'm planning to share it with you and @toliwaga and the rest of the team. Thanks.

P.S. Here is an example with the ability to restart.

[toliwaga]

@lmz  The link @bstabler provided above was broken when I moved the branch from rsg to activitysim repo.
Here is the correct link.