Offline A - Early August 2018

matt's Avatar

matt

04 Aug, 2018 04:39 AM

The first official offline beta (A, early August 2018) is available, starting with build # beta 4. Release notes are available at http://help.caltopo.com/kb/releases/offline-releases. Please post any bugs associated with the beta here, rather than in the bug reports section.

  1. 1 Posted by Adam Kelly on 06 Aug, 2018 07:20 PM

    Adam Kelly's Avatar

    When you begin a download in the new offline-maps section, is there supposed to be a traditional download prompt like when you download from whatever browser you are on or some sort of tracking progress in the command prompt?

    I have very high latency satellite internet that gets capped somewhere between 1-5mbps download speed and don't seem to have anything happen when I try to download one of the new MapBuilder offline layers.

  2. Support Staff 2 Posted by matt on 08 Aug, 2018 02:22 AM

    matt's Avatar

    The files are downloaded directly by the app without going through your browser. The download button should be replaced by constantly-refreshed status text telling you how many files and MB are left. There's no need to wait for it to complete before selecting additional files for download though.

    Are you not seeing that?

  3. Support Staff 3 Posted by matt on 11 Aug, 2018 03:57 AM

    matt's Avatar

    New and hopefully final release, build # beta 5.

  4. Support Staff 4 Posted by matt on 12 Aug, 2018 01:51 AM

    matt's Avatar

    Adam, I should have checked the beta downloader specifically. I can reproduce.

  5. 5 Posted by Adam Kelly on 13 Aug, 2018 11:46 PM

    Adam Kelly's Avatar

    Sorry that I was not able to respond to this for you quickly. I was camping without cell service until today.

  6. Support Staff 6 Posted by matt on 13 Aug, 2018 11:58 PM

    matt's Avatar

    No problem. Turns out the bug is specific to the way the javascript files are packed together, and I wasn't seeing it because even though I was testing the right code, I had a property setting to not pack the js files for easier debugging.

  7. 7 Posted by Adam Kelly on 15 Aug, 2018 10:47 AM

    Adam Kelly's Avatar

    I know that you can reproduce this download button issue but I don't know if it was addressed in the beta 5 build or not because I didn't look closely at your post date vs. build 5 release date before making this post. Regardless, this issue still persists for me with the beta 5 build (files are downloading but I am pretty much blind to their progress). I am still taking the time to download the new Elevation data, MapBuilder Overlay and MapBuilder Topo layers. I was wondering if there was really and difference between standard vs. high definition for these layers?

    Now for bugs...I have only been able to find one so far and it is going to sound pretty dumb. If I create an "arrow" shaped marker and want to position it at a certain angle (lets say 45 degrees) I can only achieve this by clicking to move the arrow's angle in the manipulation box. If I try to manually type in 45 degrees in the degrees box it will not take when I save. This appears to be working properly on the online version right now. I am not going to lie...this was my first time ever even attempting to play with the arrow icon so maybe it has always been this way?

    So far everything else for how I use SARtopo Offline seems to be working nicely. I am waiting to finish all the different offline layer downloads...hope I don't max out my GB allotment ^_^. So far though...I like this downloader much more than the previous way that was used to Dload. I wasted a lot of "GB" when Amazon's servers would time me out and my downloads would fail. The timeouts don't seem to be happening with this new downloader from what I can tell.

  8. Support Staff 8 Posted by matt on 15 Aug, 2018 03:10 PM

    matt's Avatar

    It hadn't been fixed yet, I was waiting for you to respond since I couldn't reproduce and from your initial writeup it sounded like you were looking for a browser-based file download to occur. Then I figured out about the js minimization.

    If you (or anyone) feels like the old map downloader overcounted downloads for them, let me know and I'm happy to create an exemption. The cap is in place to keep my costs in control, but while Amazon charges me per byte downloaded, it only lets me track when downloads initiate, so I can't accurately track data usage from partial downloads.

    Timeouts shouldn't be as much of an issue, but the new downloader obtains a signed URL (where the data cap is measured), and then tries that URL several times, so if a files fails partway through and then resumes, it doesn't get counted twice. The only way it should overcount is if you shut the app off with downloads still pending.

    Standard vs high definition is zoom 13 vs 14 for elevation data and 15 vs 16 for MapBuilder and USGS 7.5'. Whether there's a difference depends on your use case, but for testing remember that the new version will pull tiles from the internet if they're not available locally, and store them in a rolling cache (data/rolling.sqlite), so you need to turn your laptop's wifi off before comparing max zoom. For my uses I think standard is generally sufficient for the scanned map layers (USGS, USFS, etc) but better resolution makes a difference for the elevation layers and MapBuilder, but it depends what scale you're normally working at.

    The marker rotation issue was identified separately and will be in the next version: http://help.caltopo.com/discussions/bugs/3255-arrow-marker-orientat...

  9. Support Staff 9 Posted by matt on 16 Aug, 2018 05:04 AM

    matt's Avatar

    Beta #7 now available. Fixes the map downloader, a bug with assignment fill colors and bugs on the 104d and 104s summary pages. Also adds a final forward-compatibility tweak to the sync code, which is a little risky at this stage but hopefully won't cause any problems.

  10. 10 Posted by Adam Kelly on 20 Aug, 2018 01:57 AM

    Adam Kelly's Avatar

    So I finally have the new MapBuilder layers downloaded for my state. I am running into troubles when I put my computer into "Airplane Mode" aka turn off any data. Even though I have all of the MapBuilder stuff downloaded I can not view MapBuilder Hybrid unless I am zoomed in to almost max. MapBuilder Topo on the other hand will have tiles be completely missing at some zooms but perfect at others.

    Corrupt files? When I have data on they will start to reach out and fill in.

  11. Support Staff 11 Posted by matt on 20 Aug, 2018 03:55 AM

    matt's Avatar

    MapBuilder Hybrid requires you to have the latest year's NAIP imagery loaded as well - can you confirm that you have that?

    With MapBuilder Topo, are there any error messages on the console?

  12. 12 Posted by Adam Kelly on 20 Aug, 2018 10:47 AM

    Adam Kelly's Avatar

    I had been getting this in the console.

    WARN - Unable to read SQLite file naip_2014-1m-46-88-11.mbtiles([SQLITE_CORRUPT] The database disk image is malformed (database disk image is malformed))

    However, this tile was in the upper peninsula and the problem that I was trying to highlight in my previous post was nowhere near that tile in the lower peninsula of michigan. When I deleted and the redownloaded that tile it got rid of the error message but did not fix the issue.

    Could the problem be related to the way that the NAIP years and distributed from 2015-2014. See my images below for reference.

    Zoom Levels 16-21 load perfectly fine for the MapBuilder layers but anything zoomed out from 15 and less is where the problem occurs when in Airplane mode.

  13. Support Staff 13 Posted by matt on 20 Aug, 2018 07:19 PM

    matt's Avatar

    I just tried downloading offline data for 42.35, -84 and MapBuilder Topo and Hybrid both work for me at all zoom levels. 15/16 is the split between the "standard" and "high resolution" versions of MapBuilder, so working at >= 16 but not at <= 15 is what I'd expect if there were an issue with the standard-resolution file.

    Can you confirm that you have the following files (full directory listing included so that you can check the sizes as well) :

    -rw-r--r-- 1 matt staff 41984 Aug 20 12:08 baseline-42-84-10.mbtiles

    -rw-r--r-- 1 matt staff 1427456 Aug 20 12:08 canopy-42-84-10.mbtiles

    -rw-r--r-- 1 matt staff 15670272 Aug 20 12:08 dem-42-84-10.mbtiles

    -rw-r--r-- 1 matt staff 28837888 Aug 20 12:08 dem-9m-42-84-10.mbtiles

    -rw-r--r-- 1 matt staff 21434368 Aug 20 12:07 mapbuilder_topo-2m-42-84-10.mbtiles

    -rw-r--r-- 1 matt staff 14706688 Aug 20 12:07 mapbuilder_topo-42-84-10.mbtiles

    -rw-r--r-- 1 matt staff 44755595 Aug 20 12:08 naip_2014-2m-42-84-10.mbtiles

    -rw-r--r-- 1 matt staff 33285120 Aug 20 12:07 t-42-84-10.mbtiles

    Assuming you do, can you send me a directory listing of your data/tiles directory? Scanning each .mbtiles file on startup has proven to be too slow, so the app tries to infer their details from the filename. The trick is that it queries the metadata for the first file matching a given pattern and uses that as a template for the rest, so although I haven't had an issue with it so far, it's possible for one bad file to affect all <= 15 MapBudiler data.

  14. 14 Posted by Adam Kelly on 21 Aug, 2018 12:32 AM

    Adam Kelly's Avatar

    I am sending you my directory listing privately via email. It looks like I am definitely having some size discrepancies compared to yours...

  15. 15 Posted by nate.berkopec on 23 Aug, 2018 01:28 PM

    nate.berkopec's Avatar

    I'm not sure what hot rod juice you put in the new Beta, but our ~100GB .mbtiles install on a spinning HDD now loads up in less than 10 seconds, whereas before it could take up to 2 minutes. Absolutely loving it!

  16. Support Staff 16 Posted by matt on 23 Aug, 2018 10:21 PM

    matt's Avatar

    I had no idea spinning disk load times were so bad, until someone else mentioned it. MBTiles files downloaded from CalTopo now have their meta inferred from the filename if possible, rather than opening a connection to each file and querying from the metadata table.

  17. Support Staff 17 Posted by matt on 25 Aug, 2018 01:52 AM

    matt's Avatar

    Released as a new stable version, 4150

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac