When you begin a download in the new offline-maps section, is there supposed to be a traditional download prompt like when you download from whatever browser you are on or some sort of tracking progress in the command prompt?
I have very high latency satellite internet that gets capped somewhere between 1-5mbps download speed and don't seem to have anything happen when I try to download one of the new MapBuilder offline layers.
matt on 08 Aug, 2018 02:22 AM
The files are downloaded directly by the app without going through your browser. The download button should be replaced by constantly-refreshed status text telling you how many files and MB are left. There's no need to wait for it to complete before selecting additional files for download though.
matt on 13 Aug, 2018 11:58 PM
I know that you can reproduce this download button issue but I don't know if it was addressed in the beta 5 build or not because I didn't look closely at your post date vs. build 5 release date before making this post. Regardless, this issue still persists for me with the beta 5 build (files are downloading but I am pretty much blind to their progress). I am still taking the time to download the new Elevation data, MapBuilder Overlay and MapBuilder Topo layers. I was wondering if there was really and difference between standard vs. high definition for these layers?
Now for bugs...I have only been able to find one so far and it is going to sound pretty dumb. If I create an "arrow" shaped marker and want to position it at a certain angle (lets say 45 degrees) I can only achieve this by clicking to move the arrow's angle in the manipulation box. If I try to manually type in 45 degrees in the degrees box it will not take when I save. This appears to be working properly on the online version right now. I am not going to lie...this was my first time ever even attempting to play with the arrow icon so maybe it has always been this way?
So far everything else for how I use SARtopo Offline seems to be working nicely. I am waiting to finish all the different offline layer downloads...hope I don't max out my GB allotment ^_^. So far though...I like this downloader much more than the previous way that was used to Dload. I wasted a lot of "GB" when Amazon's servers would time me out and my downloads would fail. The timeouts don't seem to be happening with this new downloader from what I can tell.
matt on 15 Aug, 2018 03:10 PM
It hadn't been fixed yet, I was waiting for you to respond since I couldn't reproduce and from your initial writeup it sounded like you were looking for a browser-based file download to occur. Then I figured out about the js minimization.
If you (or anyone) feels like the old map downloader overcounted downloads for them, let me know and I'm happy to create an exemption. The cap is in place to keep my costs in control, but while Amazon charges me per byte downloaded, it only lets me track when downloads initiate, so I can't accurately track data usage from partial downloads.
Timeouts shouldn't be as much of an issue, but the new downloader obtains a signed URL (where the data cap is measured), and then tries that URL several times, so if a files fails partway through and then resumes, it doesn't get counted twice. The only way it should overcount is if you shut the app off with downloads still pending.
Standard vs high definition is zoom 13 vs 14 for elevation data and 15 vs 16 for MapBuilder and USGS 7.5'. Whether there's a difference depends on your use case, but for testing remember that the new version will pull tiles from the internet if they're not available locally, and store them in a rolling cache (data/rolling.sqlite), so you need to turn your laptop's wifi off before comparing max zoom. For my uses I think standard is generally sufficient for the scanned map layers (USGS, USFS, etc) but better resolution makes a difference for the elevation layers and MapBuilder, but it depends what scale you're normally working at.
matt on 16 Aug, 2018 05:04 AM
Beta #7 now available. Fixes the map downloader, a bug with assignment fill colors and bugs on the 104d and 104s summary pages. Also adds a final forward-compatibility tweak to the sync code, which is a little risky at this stage but hopefully won't cause any problems.
So I finally have the new MapBuilder layers downloaded for my state. I am running into troubles when I put my computer into "Airplane Mode" aka turn off any data. Even though I have all of the MapBuilder stuff downloaded I can not view MapBuilder Hybrid unless I am zoomed in to almost max. MapBuilder Topo on the other hand will have tiles be completely missing at some zooms but perfect at others.
Corrupt files? When I have data on they will start to reach out and fill in.
WARN - Unable to read SQLite file naip_2014-1m-46-88-11.mbtiles([SQLITE_CORRUPT] The database disk image is malformed (database disk image is malformed))
However, this tile was in the upper peninsula and the problem that I was trying to highlight in my previous post was nowhere near that tile in the lower peninsula of michigan. When I deleted and the redownloaded that tile it got rid of the error message but did not fix the issue.
Could the problem be related to the way that the NAIP years and distributed from 2015-2014. See my images below for reference.
Zoom Levels 16-21 load perfectly fine for the MapBuilder layers but anything zoomed out from 15 and less is where the problem occurs when in Airplane mode.
matt on 20 Aug, 2018 07:19 PM
I just tried downloading offline data for 42.35, -84 and MapBuilder Topo and Hybrid both work for me at all zoom levels. 15/16 is the split between the "standard" and "high resolution" versions of MapBuilder, so working at >= 16 but not at <= 15 is what I'd expect if there were an issue with the standard-resolution file.
Can you confirm that you have the following files (full directory listing included so that you can check the sizes as well) :
-rw-r--r-- 1 matt staff 41984 Aug 20 12:08 baseline-42-84-10.mbtiles
-rw-r--r-- 1 matt staff 1427456 Aug 20 12:08 canopy-42-84-10.mbtiles
-rw-r--r-- 1 matt staff 15670272 Aug 20 12:08 dem-42-84-10.mbtiles
-rw-r--r-- 1 matt staff 28837888 Aug 20 12:08 dem-9m-42-84-10.mbtiles
-rw-r--r-- 1 matt staff 21434368 Aug 20 12:07 mapbuilder_topo-2m-42-84-10.mbtiles
-rw-r--r-- 1 matt staff 14706688 Aug 20 12:07 mapbuilder_topo-42-84-10.mbtiles
-rw-r--r-- 1 matt staff 44755595 Aug 20 12:08 naip_2014-2m-42-84-10.mbtiles
-rw-r--r-- 1 matt staff 33285120 Aug 20 12:07 t-42-84-10.mbtiles
Assuming you do, can you send me a directory listing of your data/tiles directory? Scanning each .mbtiles file on startup has proven to be too slow, so the app tries to infer their details from the filename. The trick is that it queries the metadata for the first file matching a given pattern and uses that as a template for the rest, so although I haven't had an issue with it so far, it's possible for one bad file to affect all <= 15 MapBudiler data.
I'm not sure what hot rod juice you put in the new Beta, but our ~100GB .mbtiles install on a spinning HDD now loads up in less than 10 seconds, whereas before it could take up to 2 minutes. Absolutely loving it!
matt on 23 Aug, 2018 10:21 PM
I had no idea spinning disk load times were so bad, until someone else mentioned it. MBTiles files downloaded from CalTopo now have their meta inferred from the filename if possible, rather than opening a connection to each file and querying from the metadata table.