Markus Bergholz m
  • Berlin
  • Python, C, Lua, SQL, MariaDB, Redis, Ansible, IaC, Docker
  • Joined on May 07, 2018
Loading Heatmap…

m pushed to master at m/mex-mariadb

2 weeks ago

m pushed to master at m/mex-mariadb

2 weeks ago

m pushed to master at m/mex-mariadb

2 weeks ago

m pushed to master at m/mex-mariadb

2 weeks ago

m commented on issue m/openmaptiles#1

c5d instructions

# poi approach ```sql CREATE TABLE `nodes` ( `id` int(11) NOT NULL, `lon` double DEFAULT NULL, `lat` double DEFAULT NULL, `tags` varchar(8192) DEFAULT NULL, `amenity` varchar(60) GENERATED ALWAYS AS (replace(json_extract(`tags`,'$.amenity'),'"','')) STORED, PRIMARY KEY (`id`), KEY `amenity_idx` (`amenity`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 `PAGE_COMPRESSED`=1 ``` maybe needs to be partitions? ```python import mariasql from osmread import parse_file, Node import json db = mariasql.MariaSQL(host='127.0.0.1', port=3306, user='m', password='...', db='osm_nodes') skipped = 0 for node in parse_file('europe-latest.osm.pbf'): if isinstance(node, Node): if len(node.tags) > 0: tags = json.dumps(node.tags) if 'amenity' in tags or 'tourism' in tags or 'outdoor' in tags: data = dict() data['id'] = node.id data['lon'] = node.lon data['lat'] = node.lat data['tags'] = tags db.insert('nodes', data) else: skipped += 1 else: skipped += 1 printf("skipped " + str(skipped)) ```

3 weeks ago

m commented on issue m/openmaptiles#1

c5d instructions

for future Planet: 0 to 12 Europe: 13 to 14

3 weeks ago

m commented on issue m/openmaptiles#1

c5d instructions

* planet * up to level 12: 12 GB * up to level 13: 28 GB * single countries up to level 16 * germany * austria * europe up to level 14

3 weeks ago

m commented on issue m/openmaptiles#1

c5d instructions

Gen tiles part 0 parts 3 with `c5a.8xlarge` (32 cores, 64GB ram) and 1TB EBS (3000iops) ~ 1.6€ in eu-central-1 Deploy Sunday 28.06.2020 (48h + ~5h) in June + 48h + ~8h in July 4.5days

3 weeks ago

m commented on issue m/openmaptiles#1

c5d instructions

# merge mbtile files * https://github.com/mapbox/tippecanoe#tile-join `tile-join -o planet_0_13.mbtiles planet_0_12.mbtiles planet_13_13_1.mbtiles planet_13_13_2.mbtiles` # manually ```sql ATTACH 'path/to/db2.mbtiles' AS db2; INSERT OR REPLACE INTO map SELECT * from db2.map; INSERT OR REPLACE INTO images SELECT * from db2.images; INSERT OR REPLACE INTO tiles SELECT * from db2.tiles; ``` ``` .tables geocoder_data grid_key grids keymap metadata grid_data grid_utfgrid images map tiles ```

1 month ago

m pushed to master at m/dotfiles

1 month ago

m pushed to master at m/dotfiles

1 month ago

m opened issue m/dotfiles#3

nextcloud talk enterprice backend

1 month ago

m pushed to master at m/openmaptiles

1 month ago

m commented on issue m/openmaptiles#1

c5d instructions

#### hetzber sb 63 * i7-3770 (sandy bridge) * 4 cores / 8 threads * 32GB RAM DDR3 * 2x960 GB SSD * ~371.07 MB/s #### aws c5d.12xlarge * Intel(R) Xeon(R) Platinum 8275CL * 48 cores/threads * 96 GB RAM DDR??? * 2x900 GB NVME * ~2610.66 MB/s

1 month ago

m commented on issue m/openmaptiles#1

c5d instructions

try if it works and how the performance looks like. I can run it from 30th June 2020 to 1st July 2020 7-24: 17h 0-~18: 18h on largest C5d instance: * 96 cores * 4x900GB nvme (raid 0 for performance) Set threads to ~112-144?

1 month ago

m commented on issue m/openmaptiles#1

c5d instructions

https://gist.github.com/smellman/559f74158447c60ebc88c8bc12ebb505

1 month ago

m commented on issue m/openmaptiles#1

c5d instructions

https://github.com/openmaptiles/openmaptiles/issues/299#issuecomment-471438172

1 month ago

m opened issue m/openmaptiles#1

c5d instructions

1 month ago

m pushed to master at m/openmaptiles

1 month ago

m pushed to master at m/openmaptiles

1 month ago