Category: Notes

GeoServer + PostgreSQL + OpenStreetMap

To load an openstreetmap datasource in PostgreSQL/PostGIS use the following command.

# osm2pgsql –slim –username XXX –password –database streetmap ./zuid-holland-latest.osm.pbf

osm2pgsql version 0.96.0 (64 bit id space)

Password:
Allocating memory for sparse node cache
Node-cache: cache=800MB, maxblocks=12800*65536, allocation method=9
Mid: pgsql, cache=800
Setting up table: planet_osm_nodes
Setting up table: planet_osm_ways
Setting up table: planet_osm_rels
Using built-in tag processing pipeline
Using projection SRS 3857 (Spherical Mercator)
Setting up table: planet_osm_point
Setting up table: planet_osm_line
Setting up table: planet_osm_polygon
Setting up table: planet_osm_roads

Reading in file: ./zuid-holland-latest.osm.pbf
Using PBF parser.
Processing: Node(15439k 270.9k/s) Way(2349k 71.20k/s) Relation(17020 1547.27/s) parse time: 101s
Node stats: total(15439302), max(6336341363) in 57s
Way stats: total(2349653), max(676644693) in 33s
Relation stats: total(17679), max(9386429) in 11s
Sorting data and creating indexes for planet_osm_point
Stopping table: planet_osm_nodes
Sorting data and creating indexes for planet_osm_roads
Sorting data and creating indexes for planet_osm_polygon
Stopping table: planet_osm_ways
Sorting data and creating indexes for planet_osm_line
Stopping table: planet_osm_rels
Stopped table: planet_osm_nodes in 0s
Building index on table: planet_osm_ways
Building index on table: planet_osm_rels
Copying planet_osm_roads to cluster by geometry finished
Creating geometry index on planet_osm_roads
Stopped table: planet_osm_rels in 2s
Creating osm_id index on planet_osm_roads
Creating indexes on planet_osm_roads finished
All indexes on planet_osm_roads created in 3s
Completed planet_osm_roads
Copying planet_osm_line to cluster by geometry finished
Creating geometry index on planet_osm_line
Copying planet_osm_point to cluster by geometry finished
Creating geometry index on planet_osm_point
Creating osm_id index on planet_osm_line
Creating indexes on planet_osm_line finished
All indexes on planet_osm_line created in 22s
Completed planet_osm_line
Creating osm_id index on planet_osm_point
Creating indexes on planet_osm_point finished
All indexes on planet_osm_point created in 65s
Completed planet_osm_point
Stopped table: planet_osm_ways in 98s
Copying planet_osm_polygon to cluster by geometry finished
Creating geometry index on planet_osm_polygon
Creating osm_id index on planet_osm_polygon
Creating indexes on planet_osm_polygon finished
All indexes on planet_osm_polygon created in 252s
Completed planet_osm_polygon

Osm2pgsql took 355s overall
node cache: stored: 15439302(100.00%), storage efficiency: 50.00% (dense blocks: 0, sparse nodes: 15439302), hit rate: 100.00%

Create GIS database in PostgreSQL

1. Install PostgreSQL database (for example 9.6)

2. Install postgis module (for example 2.5.1)

3. Start pgAdmin 4

4. Create database

5. Enable GIS features in database with following command:
CREATE EXTENSION postgis;

6. Now shapefile can be loaded with PostGIS 2.0 shapefile tool

7. Set GEO input format (for example Rijksdriehoek)
SELECT UpdateGeometrySRID(‘wegvakken’,’geom’, 28992);

8. Create WGS84 geog column and fill it.
ALTER TABLE wegvakken ADD COLUMN geog geography(geometry);
UPDATE wegvakken SET geog = ST_Transform(geom, 4326);
CREATE INDEX wegvakken_geog_idx ON wegvakken USING GIST (geog);

9. Now pgAdmin4 can show GEO layer with OpenStreetMap layer as base.

10. Set center point of RijksDriehoek (Onze Lieve Vrouwetoren (‘Lange Jan’) in Amersfoort.)
SELECT * FROM wegvakken ORDER BY geom <-> ‘SRID=28992;POINT(155000 463000)’::geometry LIMIT 1;

11. select ST_X(geom), ST_Y(geom) from shapefile

Windows 10 upgrade takes hours

This afternoon my Quad Core 3GHZ Desktop finally received the Windows 10 creator patch. It took my desktop more than 4 hours to install it. Why is a Windows 10 patch always so slow? If i upgrade a Raspberry Pi Quad Core 1.2GHZ Linux kernel it’s only take 10 minutes. Strange that low end linux hardware upgrades are must faster than high end Windows hardware. I think that the Microsoft staff must go on training to learn how to patch Operating Systems fast and without reboots 🙂

Raspberry Pi Noobs cleanup

If you want to have a nice and clean Raspberry Pi Noobs installation. Execute te followings commands to remove unless tools:

sudo apt-get purge wolfram-engine (Reduce image size with 683MB)
sudo apt-get purge sonic-pi (Reduce image size with 151MB)
sudo apt-get purge scratch (Reduce image size with 93MB)
sudo apt-get purge scratch2 (Reduce image size with 147MB)
sudo apt-get purge scratch3 (Reduce image size with 147MB)
sudo apt-get autoclean

This initial Noobs image size will now be arround 4GB 🙂

Google Charts zoom in dataset example

google-chartsI took me a while to understand how Google Charts can be extended with a zoom/drill in data set feature. You have to add then google.visualization.events.addListener method which fetch the selected datapoint of a graph. With some extra javascript code you can reload the chart with a more detailed dataset. See example below!

<script type="text/javascript" src="https://www.google.com/jsapi"></script>
    <script type="text/javascript">
      google.load("visualization", "1", {packages:["bar"]});
      google.setOnLoadCallback(drawChart);
      function drawChart() {

      var options = {
          bars: "vertical",
          bar: {groupWidth: "90%"},
          legend: { position: "none"},
          vAxis: {format: "decimal"},
          isStacked:true,colors: ["#0066cc", "#808080"],vAxis: { format:"decimal", viewWindow: { min: 0, max: ".round($max_forecast+100)." } }, series: {
            3: {
                targetAxisIndex: 1
            }
          }
        };

        var data = google.visualization.arrayToDataTable([['','Low (kWh)','Normal (kWh)','Solar (kWh)','Forecast (kWh)'],['2006',0,0,0,0],['2007',0,0,0,0],['2008',0,0,0,0],['2009',0,0,0,0],['2010',0,0,0,0],['2011',0,0,0,0],['2012',0,0,0,0],['2013',0,0,0,0],['2014',0,0,0,0],['2015',0,0,0,0],['2016',20,20,11,278.75]]);
        var chart = new google.charts.Bar(document.getElementById("chart_div"));
        chart.draw(data, google.charts.Bar.convertOptions(options));

        google.visualization.events.addListener(chart, "select", selectHandler);

        function selectHandler(e)     {
           var year = data.getValue(chart.getSelection()[0].row, 0);
           link("pid=30&eid=88&date="+year+"-1-1");
        }
      }

function link(value)
{
	var form = document.forms['plaatenergy'];
	var newInput = document.createElement('input');
	newInput.setAttribute('type','hidden');
	newInput.setAttribute('name','token');
	newInput.setAttribute('value',value);
	form.appendChild(newInput);		
		
	form.submit();
}

Oracle SQL Loader

control file dwh.ctl

LOAD DATA
APPEND INTO TABLE CDR_UNRATED
(
WK_BMP_USAGE INTEGER EXTERNAL TERMINATED BY “,”,
WK_USAGE_TYPE INTEGER EXTERNAL TERMINATED BY “,”,
WK_EXCHANGE_ID INTEGER EXTERNAL TERMINATED BY “,”,
WK_TRANS_DT INTEGER EXTERNAL TERMINATED BY “,”,
WK_TRANS_TM INTEGER EXTERNAL TERMINATED BY “,”,
CDR_SEQUENCE_NUMBER INTEGER EXTERNAL TERMINATED BY “,”,
TRANS_DT DATE “ddmmyyyyhh24miss” TERMINATED BY “,”,
UNITS INTEGER EXTERNAL TERMINATED BY “,”,
RELATED_CALL_NUMBER INTEGER EXTERNAL TERMINATED BY “,”,
A_NUMBER CHAR TERMINATED BY “,”,
C_NUMBER CHAR TERMINATED BY “,”,
IN_SERVICE_IND CHAR TERMINATED BY “,”,
CALL_STATUS CHAR TERMINATED BY “,”,
MARKETING_CALL_STATUS CHAR TERMINATED BY “,”,
ANSWER_TIME INTEGER EXTERNAL TERMINATED BY “,”,
B_NUMBER CHAR TERMINATED BY “,”,
CHARGED_NUMBER CHAR TERMINATED BY “,”,
EXTERNAL_TYPE_ID INTEGER EXTERNAL TERMINATED BY “,”,
B_CATEGORY CHAR TERMINATED BY “,”,
SERVICE_TERMINATING CHAR TERMINATED BY “,”,
EXCEPTION_CODE CHAR TERMINATED BY “,”,
MEDIA_NUMBER CHAR TERMINATED BY “,”
)

CSV file
086,01,12345678912345,12345678912345,31122006,235959,1,31122006235959,12,0301234567,0301234567,10,U,A,5,182123451,282123451,1,15,I,abcdeabcdeabcde,1

Start SQL Loader
$ sqlldr TCT/TCT@ORCL control=dwh.ctl log=dwh_import.log data=DHW_20060501235959.csv

Oracle Application Server commands

Oracle Application Server

Show Status
$ opmnctl status

Show Cluster status
$ opmnctl @cluster status

Stop single process
$ opmnctl stopproc -process-type=XXX

Start single process
$ opmnctl startproc -process-type=XXX

Stop all processes for a cluster
$ opmnctl @cluster stopall

Start all processes for a cluster
$ opmnctl @cluster startall

———————————————

Create new instance
$ createinstance – instanceName XXXX
$ opmnctl reload
$ opmnctl startall
Note: New instance will create a new group!

Remove instance
$ removeinstance -instanceName XXXX

———————————————-

Cluster IP address = 225.0.0.2
Cluster Port Number = 6790

———————————————-

Solaris ok prompt

# OK prompt
How to get to the OK Prompt You can shutdown the system using init 0
Reboot the system and press the STOP + A keys together as the boot time messages appear on the screen
Sets diagnostic mode ( diag-switch? True ) Pressing STOP and D Keys
Sets configuration variables to default value Pressing STOP and N Keys

# Commands
OK showdevs
OK show-disks
OK boot /dev/dsk/c0t0d0s0 -s
OK boot cdrom -s

# Boot Single User Mode:
OK> boot -s

Solaris create ALOM network

sc> shownetwork
SC network configuration is:
IP Address: 0.0.0.0
Gateway address: 0.0.0.0
Netmask: 255.255.255.0
Ethernet address: 00:03:ba:57:76:8e
sc> setupsc
Entering Interactive setup mode. To exit and discard changes to that point, use Ctrl-C or to exit and save changes to that point, use Ctrl-Z.
Do you wish to configure the enabled interfaces [y]? y
Should the SC network interface be enabled [y]? y
Should the SC email alerts be enabled [y]? n
Do you wish to configure the network interface [y]? y
Should the SC use DHCP to obtain its network configuration [n]? n
Enter the SC IP address [100.100.100.100]? 192.168.0.225
Enter the SC IP netmask [255.255.255.0]? 255.255.255.240
Enter the SC IP gateway address [100.100.100.100]? 192.168.0.228
Should the TPE link test be enabled [y]? y
Do you wish to configure the managed system interface [y]? n
Do you wish to configure the network management interfaces [y]? n
Do you wish to configure the SC parameters [y]? n
Your ALOM configuration profile has been successfully completed. To activate your network configuration, please reset the SC.
sc> resetsc
Are you sure you want to reset the SC [y/n]? y
User Requested SC Shutdown

SC Alert: SC System booted.

SC Alert: AC power unavailable for PSU @ PS0.

[SC Status 7fff – OK] Enter #. to return to ALOM.
Oct 18 14:16:57 nlnwhabcm03 rmclomv: SC unretrieved msg OCT 18 13:16:54 2004 UTC [SC System booted.]

Copyright 2003 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.

Sun(tm) Advanced Lights Out Manager 1.3 (nlnwhabcm03)

Please login: admin
Please Enter password: ******

sc> shownetwork
SC network configuration is:
IP Address: 192.168.0.225
Gateway address: 192.168.0.228
Netmask: 255.255.255.240
Ethernet address: 00:03:ba:57:76:8e
sc>

# Open console by ALOM port

$ telnet db01-alom
– login: admin
– password: admin

$ console -f
– login: root
– password: <>

Now your are login!

Solaris cluster commands

hastatus (Show cluster status)
hares -offline -sys (Stop component)
hares -online -sys (Start component)

hagrp -clear (Clear groep die gecrashed is)

hastatus -sum # Show autodisable groups
hagrp -autoenable

vxprint – Show status Veritas

# Check Veritas Cluster main.cf file
hacf -verify /etc/VRTSvcs/conf/config -display

# Configuration Read /Write
haconf -makerw
hares -modify DB_OP_XX_SID Critical 0
hares -value DB_OP_XX_SID Critical
hares -modify DB_OP_XX_SID Enabled 0
hares -value DB_OP_XX_SID Enabled
haconf -dump -makero

SUN StorEdge Backup Solution scripts

# General
/usr/sbin/nsr/nsrim -v (Print Volume information of all used tapes)

# ——————————————————————————
# BACKUP (SINGLE TAPE DEVICE)
# ——————————————————————————

0 19 * * 1-5 /usr/sbin/nsr/tape_label_mount.sh >> /nsr/logs/tape.log
#!/bin/sh
#
# Load / relabel / mount tape
#
/usr/sbin/nsr/nsrmm -l -R << EOF y EOF sleep 30 /usr/sbin/nsr/nsrmm -m 20:30 start total backup StorEdge backup software regeld dit! 0 6 * * 1-5 /usr/sbin/nsr/tape_unmount.sh >> /nsr/logs/tape.log
#!/bin/sh
#
# unLoad /eject tape
#
/usr/sbin/nsr/nsrmm -j

# ——————————————————————————
# BACKUP (TAPE ROBOT 8 POSITIES)
# ——————————————————————————

00:30 start load/relabel/mount script

#!/bin/sh
#
# Load / label / mount tape
#
# Volume Name Tape Location Note
# volume.001 Location 1 Monday
# volume.002 Location 2 Tuesday
# volume.003 Location 3 Wedneyday
# volume.004 Location 4 Thursday
# volume.005 Location 5 Friday
# volume.006 Location 6 Saterday
# volume.007 Location 7 Sunday
# Location 8 Clean tape location

DAYOFWEEK=`date +”%u`

# Load / relabel tape (-Y No approve of action is needed)
/usr/sbin/nsr/nsrjb -Y -L -S ${DAYOFWEEK} svolume.00${DAYOFWEEK}

# Mount tape
/usr/sbin/nsr/nsrjb -l -S ${DAYOFWEEK}

01:00 start total backup
StorEdge backup software regeld dit!

23:30 start unmount / unmount script

#!/bin/sh
#
# unLoad tape
#

DAYOFWEEK=`date +”%u`

# unmount / unload tape
/usr/sbin/nsr/nsrjb -u -S ${DAYOFWEEK}