Problem extracting OSRM data using Docker - docker

Question

I am trying to extract OSRM data but having troubles. In a previous question (lua error when trying to extract osrm data), I was encountering an error. So I am not trying to do the same thing except with Docker. I am using a newer Mac with a Silicon chip. I first installed Docker (https://docs.docker.com/desktop/install/mac-install/; and then in the Terminal docker pull osrm/osrm-backend). I download the North America data from https://download.geofabrik.de/north-america.html. I have allocated 50GB of memory to the instance as well as several cores. If, in Terminal, I run docker run -t -v /Users/[name]/osrm-backend/docker:/data osrm/osrm-backend osrm-extract -p /opt/car.lua /data/north-america-latest.osm.pbf, I get:
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
[info] Parsed 0 location-dependent features with 0 GeoJSON polygons
[info] Using script /opt/car.lua
[info] Input file: north-america-latest.osm.pbf
[info] Profile: car.lua
[info] Threads: 5
[info] Parsing in progress..
[info] input file generated by osmium/1.14.0
[info] timestamp: 2023-01-05T21:21:39Z
[info] Using profile api version 4
[info] Found 3 turn restriction tags:
[info] motorcar
[info] motor_vehicle
[info] vehicle
[info] Parse relations ...
TBB Warning: Exact exception propagation is requested by application but the linked library is built without support for it
terminate called after throwing an instance of 'tbb::captured_exception'
what(): PBF error: truncated data (EOF encountered)
qemu: uncaught target signal 6 (Aborted) - core dumped
The process might still be running (meaning that I can't execute any new commands), but hasn't apparently done anything for quite a while.
What do these warnings mean, and how do I fix them?
Note also that if I run another file, it works. docker run -t -v /Users/[name]/osrm-backend/docker:/data osrm/osrm-backend osrm-extract -p /opt/car.lua /data/us-virgin-islands-latest.osm.pbf
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
[info] Parsed 0 location-dependent features with 0 GeoJSON polygons
[info] Using script /opt/car.lua
[info] Input file: us-virgin-islands-latest.osm.pbf
[info] Profile: car.lua
[info] Threads: 5
[info] Parsing in progress..
[info] input file generated by osmium/1.14.0
[info] timestamp: 2023-01-05T21:21:39Z
[info] Using profile api version 4
[info] Found 3 turn restriction tags:
[info] motorcar
[info] motor_vehicle
[info] vehicle
[info] Parse relations ...
[info] Parse ways and nodes ...
[info] Using profile api version 4
[info] Using profile api version 4
[info] Using profile api version 4
[info] Using profile api version 4
[info] Parsing finished after 4.90155 seconds
[info] Raw input contains 1316532 nodes, 72554 ways, and 97 relations, 30 restrictions
[info] Collecting way information on 30 restrictions...ok, after 0.000596s
[info] Collecting way information on 0 maneuver overrides...ok, after 0.000106s
[info] Sorting used nodes ... ok, after 0.004332s
[info] Erasing duplicate nodes ... ok, after 0.000402s
[info] Sorting all nodes ... ok, after 0.002219s
[info] Building node id map ... ok, after 0.00384s
[info] Confirming/Writing used nodes ... ok, after 0.111959s
[info] Writing barrier nodes ... ok, after 0s
[info] Writing traffic light nodes ... ok, after 0s
[info] Processed 73181 nodes
[info] Sorting edges by start ... ok, after 0.006168s
[info] Setting start coords ... ok, after 0.010829s
[info] Sorting edges by target ... ok, after 0.006618s
[info] Computing edge weights ... ok, after 0.111005s
[info] Sorting edges by renumbered start ... ok, after 0.01132s
[info] Writing used edges ... ok, after 0.013881s -- Processed 75963 edges
[info] Writing way meta-data ... ok, after 0.001686s -- Metadata contains << 9487 entries.
[info] Collecting node information on 0 maneuver overrides...ok, after 8e-06s
[info] Collecting node information on 30 restrictions...ok, after 0.000959s
[info] writing street name index ... ok, after 0.004272s
[info] extraction finished after 5.21655s
[info] Generating edge-expanded graph representation
[info] . 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%
[info] Node compression ratio: 0.19777
[info] Edge compression ratio: 0.220645
[info] graph compression removed 8425 annotations of 9487 in 0.014951 seconds
[info] Find segregated edges in node-based graph ...
[info] ok, after 0.426069s
[info] Segregated edges count = 57
[info] Writing nodes for nodes-based and edges-based graphs ...
[info] Geometry successfully removed:
compressed edges: 33300
compressed geometries: 150658
longest chain length: 151
cmpr ratio: 0.22103
avg chain length: 4.52426
[info] Removing invalid restrictions...removed 0 invalid restrictions, after 0.000302s
[info] Constructing restriction graph on 30 restrictions...ok, after 0.001196s
[info] Generating edge expanded nodes ...
[info] . 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%
[info] Expanding via-way turn restrictions ...
[info] . 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% 100%
[info] Generated 32790 nodes (5 of which are duplicates) and 75329 segments in edge-expanded graph
[info] Generating edge-expanded edges
[info] . 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% .
[info] Sorting and writing 0 maneuver overrides...
[info] done.
[info] Renumbering turns
[info] Writing 0 conditional turn penalties...
[info] Generated 75329 edge based node segments
[info] Node-based graph contains 32735 edges
[info] Edge-expanded graph ...
[info] contains 61286 edges
[info] Timing statistics for edge-expanded graph:
[info] Renumbering edges: 0.001581s
[info] Generating nodes: 0.01637s
[info] Generating edges: 0.7993s
[info] Generating guidance turns
[info] . 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% .
[info] done.
[info] Created 26 entry classes and 1550 Bearing Classes
[info] Handled: 0 of 13 lanes: 0 %.
[info] Assigned 68762 turn instruction types:
[info] new name: 3076 (4.47%)
[info] continue: 5507 (8.01%)
[info] turn: 27959 (40.66%)
[info] merge: 21 (0.03%)
[info] on ramp: 5 (0.01%)
[info] off ramp: 8 (0.01%)
[info] fork: 832 (1.21%)
[info] end of road: 10601 (15.42%)
[info] notification: 4 (0.01%)
[info] enter roundabout: 6 (0.01%)
[info] enter roundabout turn: 11 (0.02%)
[info] (noturn): 4261 (6.20%)
[info] (suppressed): 16403 (23.85%)
[info] exit roundabout: 8 (0.01%)
[info] exit roundabout turn: 11 (0.02%)
[info] (stay on roundabout): 19 (0.03%)
[info] (sliproad): 30 (0.04%)
[info] Assigned 68762 turn instruction modifiers:
[info] uturn: 5105 (7.42%)
[info] sharp right: 2730 (3.97%)
[info] right: 16456 (23.93%)
[info] slight right: 2962 (4.31%)
[info] straight: 19307 (28.08%)
[info] slight left: 2989 (4.35%)
[info] left: 16526 (24.03%)
[info] sharp left: 2687 (3.91%)
[info] Guidance turn annotations took 1.18582s
[info] Writing Intersection Classification Data
[info] ok, after 0.00886s
[info] Writing Turns and Lane Data...
[info] ok, after 0.01241s
[info] Saving edge-based node weights to file.
[info] Done writing. (0.005628)
[info] Computing strictly connected components ...
[info] Found 64 SCC (2 large, 62 small)
[info] SCC run took: 0.00466692s
[info] Building r-tree ...
[info] Constructing r-tree of 75329 segments build on-top of 73181 coordinates
[info] finished r-tree construction in 0.149264 seconds
[info] Writing edge-based-graph edges ...
[info] ok, after 0.121746s
[info] Processed 61286 edges
[info] Expansion: 26516 nodes/sec and 11881 edges/sec
[info] To prepare the data for routing, run: ./osrm-contract "/data/us-virgin-islands-latest.osrm"
[info] RAM: peak bytes used: 183009280

Answer 1

It might be a memory issue, try to allocate more. As somebody mentioned in this issue https://github.com/Project-OSRM/osrm-backend/issues/5614#issuecomment-560504788:
The only real reference point I have is that it takes about 100GB of
RAM to process North America.

Answer 2

Try downloading the file /data/north-america-latest.osm.pbf again or verify if it downloaded successfully.
It could be corrupt due to a broken download.
The error you are facing seems to match here - https://github.com/peter-evans/osrm-backend-k8s/issues/48