Update lookups

[afischerdev]

This is the call for a bigger update on lookups.dat

The Android apk doesn't need an update for that. Lookups.dat and profiles are updated automaticly.
Next apk update will then contain the new versions.

So only web services are affected.

And an update for the profiles is needed when there is an interest on man_made=pier or other changes.

Please see #416 for more.

[zod]

If we generate new segments using this update lookups.dat all users must update their segment files as soon as they get the new lookup.dat. Correct?

• I'm not to sure about compatibility. When do we need to increase the major version?
• I don't think we need hiking-beta.brf in the repository, but just as downloadable profile.

[afischerdev]

If we generate new segments using this update lookups.dat all users must update their segment files as soon as they get the new lookup.dat. Correct?

Yes. When the server update is done and an apk user updates the data lookups.dat and profiles are downloaded first then rd5 download starts. But when user has called e.g. only one rd5 but has 4 on board he has a problem when next enter an outdated rd5 - he will get a notice.
We already discussed if we drop the others or not.

I'm not to sure about compatibility. When do we need to increase the major version?

When index changes a new number is needed. Smaller changes like aliases don't change it.

I don't think we need hiking-beta.brf in the repository, but just as downloadable profile.

Yes, until new apk version is out. Current looks for hiking-beta
But we could exclude it from profile zip

[polyscias]

I think breaking things for (many) users is bad and like I wrote in #416, I see quite some pain for removing oneway=true.

I did post PR to update the profiles of @poutnik, see poutnikl/Trekking-Poutnik/pull/30 but that has not been merged as of now.

Apart from the profiles of @poutnik it also impacts most other profiles so I think it is better to keep it in the profile for now, but let's remove it for the brouter profiles.

But when user has called e.g. only one rd5 but has 4 on board he has a problem when next enter an outdated rd5 - he will get a notice.

How does that notice look like?
I run my tests with the server but there the error reporting is not good, the error is only logged at the server side.

Also, what about return meta.lookupVersion == 10;

For the remainder the patch looks good to me, I can run later this weekend a map generation run with it and give some stats.

[nrenner]

Quoting @polyscias from #416 (comment):

For tracktype=4, can we added "grade3-5" as alias, that is mapped 917 times

I wondered where the tracktype=grade3-5 value comes from.

The OSM tag history graph (see also Taginfo chronology) looks suspicious:

And indeed, there are two unique uses:

• 945 ways added by an import in 2014 with changeset 21660337 for some region in Norway [1]
• 538 ways added since July by a single paid mapper from Kaart [2], almost all to highway=residential in South Africa

The latter and other sporadic edits are all using the iD editor, which actually has that value in the list of suggestions when using the properties table, which is most probably because of the value count of that single import.

Therefore I think the grade3-5 alias should not be included, such an endorsement would only further propagate the use of this undocumented and unsupported value.

---

[1]

curl -o changeset-21660337.osc https://www.openstreetmap.org/api/0.6/changeset/21660337/download
grep "grade3-5" changeset-21660337.osc | wc -l

[2] https://overpass-turbo.eu/s/1kNX

[polyscias]

Nice analysis @nrenner!

I am perfectly fine with removing the grade3-5 alias but my opinion is, also with knowing this story, such an alias is fine.

There are more tags in lookups.dat that are undocumented, for instance noexit=yes in way context and junction=approach

[poutnikl]

I did post PR to update the profiles of @poutnik, see poutnikl/Trekking-Poutnik/pull/30 but that has not been merged as of now.

I have somehow missed the PR and related discussion. I will try to address it and generate new profiles, but after Aug 15, due vacation.

Another delay will be for LocusMap users, with profiles embedded to application or online LocusMap client. They use profiles from several sources.

[polyscias]

I have somehow missed the PR and related discussion. I will try to address it and generate new profiles, but after Aug 15, due vacation.

That would be nice.

Another delay will be for LocusMap users, with profiles embedded to application or online LocusMap client. They use profiles from several sources.

Do you have some more information on this, in particular the exact profiles that are used? I like to add them to my collection of profile that I check for impact of this update.

I can run later this weekend a map generation run with it and give some stats.

With quite some delay I did run it and looking at all the .rd5 files generated versus those on http://brouter.de/brouter/segments4/
I see a mix of segments that did increase in size and decrease in size but net effect is that the overall size is -0.04%

The extremes:

W90_N30.rd5   4998766   34854799 +14.34%
E25_S10.rd5    477592    3252900 +14.68%
E45_S30.rd5    113831     768699 +14.81%
..
E10_N65.rd5   -505069    3792746 -13.32%
E15_N65.rd5   -952119    7259065 -13.12%
E10_N60.rd5  -3225188   23634055 -13.65%

I am planning to get the updated lookup.dat, profiles and one .rd5 on my mobile and see what it does.

[polyscias]

See also brouter-web#669, can we add also:

• railway=platform
• public_transport=platform
• man_made=pier

The first two are accessible on foot and a pier is often a way to access a ferry both on foot and by bicycle.

[Luen]

I'm wondering if we can add 'stream' in the profile. Currently, we can route via waterway=river but not streams.
gpxstudio/gpx.studio#166

See also https://wiki.openstreetmap.org/wiki/Tag:waterway=stream

[afischerdev]

Added T1-hiking again to the lookups.dat. It is not used anymore but the hiking-mountain profile contains it.
That is not the problem - we will change the inbuilt profile -, but I see childs of this profile driven by user ideas will break.

So I followed @polyscias

I think breaking things for (many) users is bad

And a check of the recently added tags/values ​​was also performed.
With the last small resort we have now a total for the same Germany export with 250K bytes less.

[afischerdev]

Please do not forget this test for your profiles

java -cp brouter.jar  btools.expressions.IntegrityCheckProfile profiles2/lookups_my.dat profiles2

[afischerdev]

The values yes and designated already have seperate entries, so it is not necessary to merge them.
Size check old and new shows equal sizes.

[ProBackup-nl]

@afischerdev I don't like the alias for the traffic_calming choked_table in lookups.dat. A choked table is not a synonym for a table. In real world the choked table is the extra annoying combination of a choker and a table. With a normal choker without traffic, you have the luck that you can pass at full speed. With the choked_table without other traffic, you still need to reduce speed because of the vertical deviation. In a good profile the choked_table needs to have a penalty of approximately the table plus the choker.

[afischerdev]

@ProBackup-nl
Thanks for the additional info.
At the moment we have this nodes (new lookup with counts for Germany, alias included)

traffic_calming;0000011529 island
traffic_calming;0000008972 choker
traffic_calming;0000007656 bump
traffic_calming;0000007408 chicane
traffic_calming;0000006360 table
traffic_calming;0000004850 hump
traffic_calming;0000001841 cushion
traffic_calming;0000000543 rumble_strip
traffic_calming;0000000368 dip
traffic_calming;0000000273 yes

Looks a little bit like the traffic_calming icons.
But we are interested in routing data not in mapping data. We need to know that there comes a speed breaker and may be it is a hard or more a soft format or only a narrowed road.
I think we don't need all this types.

You are very interested and committed to this topic. My be you like to do more investigation how to reduce this types and classify the types by penalty.
And I noticed there are lots of way with this tag. We have to collect them as well.

[xuiqzy]

@afischerdev In case some of the traffic_calming values are to be combined, I wanted to note that some differences like cushion vs hump might not be important for cars but are important for bicycles and motorcycles (can pass on the side) as well as buses/emergency vehicles (pass over it without a slowdown). So it would be great to keep those differences available for routing profiles if possible.
If these considerations are already clear, sorry for the noise!

[ProBackup-nl]

We need to know that there comes a speed breaker and may be it is a hard or more a soft format or only a narrowed road. I think we don't need all this types.

May be you like to do more investigation how to reduce this types and classify the types by penalty. And I noticed there are lots of way with this tag. We have to collect them as well.

What we need to know, depends on the type of transportation: by foot, horse, boat, train or MTB the traffic calming relevance is absent, on a slow bicycle a little near absent. More relevant is traffic calming when using a velomobile or moped, and those are different compared to a car or wide truck.

The iD editor has 12 tags translated into Dutch, of which 8 tags contain an icon. Amazing is that from the brouter profiles2 only gravel and quaelnix-gravel use traffic_calming=* (one penalty for any value). The frequency of travel_calming in NL and riding a velomobile without suspension lead me to improving the vm-touristic profile.

For me personally the traffic_calming order of importance:

1. is it a bump or double_dip? No Go -> possible velomobile damage
2. because double_dips are sometimes mistagged as 2 seperate dips, I will avoid the dip too.
3. is it full street width or can I pass by on the main road, without a change of surface? F.e. cushions can often be passed by when riding on a motor bike or velomobile.
4. is the vertical deviation partly made out of preformed, often concrete, element(s), especially the ramps? Yes, then trapezoid type = horrible; otherwise its more likely a sinusoidal type, which is better acceptable.
5. lateral deviation is important too, chicane != choker
6. both lateral and vertical deviation choked_table is a different type.

[afischerdev]

@xuiqzy @ProBackup-nl
Agree with all, but we are talking not about profiles.
Here the context is lookups.dat And the entries must give the possibility for the profile to use it in every case that is needed.
So what is your suggestion for an entry list?

traffic_calming;0000000000 main_value list of alias

[ProBackup-nl]

For node:traffic_calming (without much thought about aliases):

1. yes
2. bump
3. chicane
4. choked_table
5. choker
6. dip
7. double_dip
8. table
9. rumble_strip

no|island|rumble_strip|painted_table

For node:surface:

1. asphalt
2. paving_stones|paved| sett| cobblestone|anything with a textured surface (not concrete)
3. all others

For bicycle profiles it's handy to know if there is a bicycle_bypass=yes, that would reduce the penalty. Tagged 324 times in a full NL bbox, 645 times in Europe.

[ProBackup-nl]

@afischerdev I updated the comment above, with traffic_calming number 9 rumble_strip, and surface number 2 paving_stones. The reason is why I don't prefer to cycle to some villages cognitively without active memory why. After looking back the images I saw the good asphalt was cut open at 12 different spots within 500 meter and filled with paving_stones. Each strip individually not enough to mark, 5-10 meters, some with a slight vertical deviation, not enough for a hump. All 12 tiny road surface differences combined in 45 seconds become one big annoying new traffic calming thing.

[dadebue]

Hi, noob question here: I want to have the via_ferrata tags information. I can't just go ahead and use the new lookups with the current rd5 files right? From the developer guide:

The lookup-table is allowed to grow over time, to include more tags
and values as needed. To support that evolution, it carries a major
and a minor version number. These numbers are also encoded into
the routing data files, taken from the lookups.dat that is used
to pre-process the routing data files.

[RuHitt]

Hello everyone,
When can we expect the new 'lookups.dat' file to be included in a release? This would be really helpful for using 'new' tags in routing files. This would be especially useful for evaluating 'cycleway:both=lane', as this tag is now used exclusively for 'cycle paths'. You could argue that these are not actually cycle paths, but nevertheless, my current profile cannot route through these roads appropriately.
Is there any way I can help speed up this process?

Kind regards, Ruben