Saturday, 11 February 2017

RESTFull GEOJson with Swagger and Slim

I often find myself having to write RESTFull api for small project.  So I have search and tried several tools. I finally choose to use Swagger to write the documentation and Slim micro framework to implements it.

The first step is writing the yalm file in the Swagger editor (there is a nice tutorial at Api Handyman blog). With the yalm specification file it's simple to build an html documentation. Swagger have two pre build schema in html (html and html). You can export it, from Swagger Editor with Generate Client button (where you can export other languages client code).

 To create the Slim skeleton, simply click on the Generate server button and choose Slim. Now you have two zip file that contains documentation and the skeleton code.

Usually my project folder is composed by three subfolders: app (with the api code), api_doc (with the documentation generated by Swagger) and then an db_doc (the database documentation with diagram).

To provide a geographic information the request retrive or send a GEOJson, so I have write a  yalm gist as example.

To store the data I use PostgresSQL/PostGIS, using this query to retrive the data:
 
 SELECT row_to_json(featuresCollection)
          FROM ( SELECT 'FeatureCollection' As type,
          array_to_json(array_agg(features)) As features
          FROM (SELECT 'Feature' As type
         ,ST_AsGeoJSON(data.the_geom, 4)::json As geometry
         , row_to_json((SELECT prop FROM (SELECT property1, property2.....) As prop         )) As properties
         FROM myTable As data ) As features ) As featuresCollectionc;

Saturday, 14 January 2017

A leaflet template for ionic

Leaflet is a very useful javascript library to insert map into an app. It has lots of extensions, which simplify it's use. I use it mainly in ionic's app, so I have created a repo on GitHub as a start point. It's extends the side menu template with leaflet.

The key points are:
  1. Get a GeoJson through an api call (markersApi.js)
  2. Create a map (with OSM as baselayer).
  3. Add the GeoJson to the map (using the marker-cluster library).
  4. Add a marker on tap.

Add the map to the html

To add leaflet map in ionic (and angular) there is a directive to simplify it: angular-leaflet-directive (or you can use ui-leaflet). In the html you can just put:

<leaflet data-tap-disabled="true" defaults="map.defaults" id="map" layers="map.layers" lf-center="map.center"></leaflet>


And after you have incluse in the app's model dependencies, the angular-leaflet directive, and have defined an object in the controller as $scope.map which contains the center point when load the map, the layer to load and some optional parameter you can see the map.
 
//set the property of the map
$scope.map = {
             center: {
                lat: 46,
                lng: 11,
                zoom: 8
             },
             defaults: {
                zoomControl: false,
                attributionControl: true
             },
             layers: {
                baselayers: {
                    xyz: {
                        name: 'OpenStreetMap (XYZ)',
                        url: 'http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png',
                        type: 'xyz',
                        zoomControl: false,
                        layerOptions: {
                            attribution: '© OpenStreetMap contributors'
                        }
                    }
                }
            }
        };

You also can center the map to the actual position:
 
var options = {enableHighAccuracy: false,timeout: 20000};

//center the map on the actual position.

var setCurrentPosition = function () {
   $ionicLoading.show({
   template: '',
   duration: 7000
   });
   $cordovaGeolocation.getCurrentPosition(options).then(function (position) {
      if (position) {
         $scope.map.center.lat = position.coords.latitude;
         $scope.map.center.lng.longitude = position.coords.longitude;
         } else {}
       }).catch(function (err) {}).finally(function () {
         $ionicLoading.hide();
       });
};

Add markers to the marker

The markers are in a GeoJson format, so they are among throughout the L.GeoJSON layer:
 
//create a marker cluster layer
var markers = L.markerClusterGroup();

/*
*When the view enter:
* get the map.
* get the markers form the api.
* create the geojson layer
* add the geojson layer to the marker cluster layer
* add the marker cluster layer to the map
*
*/
$scope.$on("$ionicView.enter", function (scopes, states) {
   leafletData.getMap("map").then(function (map) {
   map.invalidateSize();
   markersApi.getPoints().then(function (resp) {
      var geojson = new L.GeoJSON(resp, {
          
         onEachFeature: function (feature, layer) {
            //open a ionic modal on click 
            layer.on('click', function (e) {
            $scope.map.center.lat = feature.geometry.coordinates[1];
            $scope.map.center.lng = feature.geometry.coordinates[0];
            $scope.feature = feature;
            $scope.openLayersModal();
            });
           }
      });
   markers.addLayer(geojson);
   map.addLayer(markers);
}, function (err) {
      console.log('ERROR', err.status);
});
});

Add a marker on tap

To add a marker on tap,  you can add this code:

var setMarker = function (latlng) {
   if (!newMarker) {
     newMarker = new L.marker(latlng).addTo(map);
   } else {
      newMarker.setLatLng(latlng).update();
   }
};
var newMarker = null;
map.on('click', function (e) {
setMarker(e.latlng);
});



the repo is available on GitHub at this link.

NOTE: angular-leaflet use leaflet at version 0.7.7 so the marker-cluster is the old version as you can see in the bower.json file.

Monday, 2 January 2017

My notes on JavaScript

I started study JavaScript about one years ago, I studied the basic rule and then, due to my work, I moved quickly to use some framework (Ionic and Angular) so I omitted to study it in depth and also  the best practise... Because of this bad start, this December, I re-started to study and I'm writing some notes (mainly on different things than other languages).

Variables

  • keyword const, to declare a constant.
  • Number:
    • Number.NEGATIVE_INFINITY<Number.MIN_VALUE<Number.MAX_VALUE<Number.POSITIVE_INFINITY.
    • Math is an "object" with the most common mathematic functions.
    • Some useful method of Number: parseInt(), parseFloat(), isNaN(), isInfinite().
  • Multiline string, useful to include variable values into the string:
        var a=2;
        var b=2;
        var myString=`this is
                      my value ${a+b} `;

The if statement

  • To test if an array is empty, simply use if(myArray.length) not if(myArray.length>0).
  • To test if a string is void use if(myString).
  • Test only if an object is null if(myObj===null) not if it's null and undefined.

The scope

The scope is a "nested chain": the variable definition is searched in the nested scope, if there isn't then it's search in the upper scope.

Functions

The functions parameters are stored in an object (arguments), similar to an array, so it's possible to retrive parameters with: arguments[i].

Collections

  • Array
    • it can stores several type.
    • it's a map, the key is the index. The length() method return the last index plus 1, so 
          var a=2;
          var myArray=[];
          myArray[25]="myLastObject";
          console.log(myArray.length()); //print 26
      
    • Usefull methods: foreach, concat, join, pop, push, shift, unshift, reverse, sort.
  • Maps, only in ES6.
  • Sets, only in ES6.

Tools

  • bower, package manager.
  • npm, pakage manager.
  • javascriptlint
  • gulp, builder, it's possible run several task (lint, minify resources..).

Saturday, 16 July 2016

Notes about SQLite

I'm using SQLite in several project, mainly in mobile app (Android SDK or with Cordova plugin).
SQLite is the best solution to store data locally, but when using it?  Maybe it's better to understand when does't use it:

IF data are detached from the app OR there are lots of concurrent writers OR Big Data

THEN t's better build a server side services (client-server)

OTHERWISE  SQLite


Main Properties:

  1. self-contained
  2. serverless
  3. zero-configuration
  4. transactional: Atomic,Consistent,Isolated, Durable (ACID)

SQLite han't all feature that usually a server side db (as MySQL) have:

  • Right and full outer join.
  • ALTER TABLE isn't completely supported.
  • There isn't Date and Time DataType but it's possible to store data as a REAL (Julian day number), INTEGER (Unix time) or TEXT (ISO8601) and extract it by specific function. I prefer store date as text so for example the today's date is strftime("%s",'now') or use in a query as "strftime('%Y-%m-%d',DATA_RILIEVO) DESC" where DATA_RILIEVO is the columns name.
  • If an INTEGER column is a primary key then it's an alias of the ROWID column. ROWID set is value to first available integer, otherwise it's possible to use AUTOINCREMENT.
  • I have noticed that multiple primary key isn't always supported.


reference

site SQLite



Thursday, 31 December 2015

Chart.js 2.0 in Ionic

There are lots JS data chart libraries... it's hard to choose one that's right for you. I started using chartjs because it's responsiveness, it has good support for users interactions and there is an angular directives that support it: angular-chart.js. Angulars directive support chart.js 1.0, after some attempts, I have had some issues:
  1. Time series aren't supported.
  2. It isn't simple to manage missing value.
Then I moved to chart.js 2.0, which was under development. Now, there is a beta version of the library.

So, as first step I have installed it with bower (at https://github.com/nnnick/Chart.js.git#v2.0-dev), and then I try to put it into a Ionic project.
I have put the chart inside a slider:

    <ion-nav-buttons side="left">
        <button class="button button-icon button-clear ion-navicon" menu-toggle="left">
        </button>
    </ion-nav-buttons>

    <ion-content>
        <ion-refresher on-refresh="doRefresh()" pulling-text="Pull to refresh...">
        </ion-refresher>

        <ion-slide-box animation="slide-left-right-ios7" show-pager="true">
            <ion-slide>
                <div class="list card">
    <div class="item">
      First series
                  </div>
    <div class="item item-body">
      <canvas class="chart" height="300" id="firstPlot"></canvas>
    </div>
       </div>
   </ion-slide>
        <ion-slide>
   <div class="list card">
     <div class="item">
  Second
            </div>
     <div class="item item-body">
       <canvas class="chart" height="300" id="secondPlot"></canvas>
            </div>
 </div>
      </ion-slide>
   </ion-slide-box>
  </ion-content>
 </ion-view>
And the controller code:
     
angular.module('myModule', [])controller('chartCtrl', function ($scope, $ionicSlideBoxDelegate,
 $stateParams, $sce , getMyDataServices) {
    Chart.defaults.global.pointHitDetectionRadius = 1;
    Chart.defaults.global.datasetFill = false;
    /*dataToPlot contains 2 series chart1 (which is an array of dimension 3xn => [[],[],[]])
    / and chart2 which is an array 1xn
    / dataToPlot.data contains the labels for x axis.
    */
    var dataToPlot=getMyDataServices.getDataToPlot();
    var chart1=null;
    var chart2=null;
    //if I want to plot an interval.
    var l = dataToPlot.data.length - 1;
    var start = 0;
    //if I want to formatting the data
    var labels = dataToPlot.data.map(function (d) {
                return d;
            });
    chart1 = createPlot(chart1, {series: [dataToPlot.chart1], serieLabel: ["Plot1 "],
                                labels: labels, 
                                colors:['rgb(0, 0,255)','rgb(0, 0, 0)', 'rgb(255, 0, 0)']},
                                start, end, "#plot1a", "[-]");
    chart2 = createPlot(chart2, {series: [dataToPlot.chart2], serieLabel: ["Plot2 "],
                        labels: labels, colors: ['rgb(0, 0, 255)']},
                         start, end, "#plot2", "[-]");

    //update
    $scope.$broadcast('scroll.refreshComplete');
    $ionicSlideBoxDelegate.update();
    $scope.$broadcast('scroll.refreshComplete');

    $scope.doRefresh = function () {
        getData();
        $ionicSlideBoxDelegate.update();
    };
});


var opt = {
    tooltips: {
        mode: 'label'
    },
    hover: {
        mode: 'label'
    },
    scales: {
        xAxes: [{
                display: true,
                scaleLabel: {
                    show: true
                }
            }],
        yAxes: [{
                display: true,
                scaleLabel: {
                    show: true,
                    labelString: "void"
                }
            }]
    }, elements: {
        line: {
            borderCapStyle: 'round',
            borderWidth: 1.0,
            tension: 0.1
        },
        point: {
            backgroundColor: Chart.defaults.global.defaultColor,
            borderWidth: 1,
            borderColor: 'rgb(0,0,0)',
            hitRadius: 10,
            hoverRadius: 5,
            hoverBorderWidth: 1,
            radius: 2
        }
    }
};



var createPlot = function (plot, dataToPlot, start, end, id, scaleLabel) {
    opt.scales.yAxes[0].scaleLabel.labelString = scaleLabel;
    var dataset = [];
    if (!!dataToPlot.series) {
        for (var i = 0; i < dataToPlot.series.length; i++) {
            if (!!dataToPlot.series[i]) {
                dataset.push({
                    label: dataToPlot.serieLabel[i],
                    data: dataToPlot.series[i].slice(start, end).map(function (x) {
                         //parse the value to int, only for me                   
                          return parseInt(x, 10);
                    }),
                    fill: false,
                    borderColor: dataToPlot.colors[i]
                });

            }
        }
        var config = {
            type: 'line',
            data: {
                labels: dataToPlot.labels.slice(start, l),
                datasets: dataset
            },
            options: opt
        };
        if (plot !== null) {
            plot.destroy();
        }
        var ctx = angular.element(document.querySelector(id));
        plot = new Chart(ctx, config);

        return plot;
    }
};



Friday, 10 April 2015

A RESTFul with Openshft, Python + Flask and AeroGear

As a new year's resolution I have  started to learn Python. So I have read an introduction to this programming language. Then as next step I have created a RESTful API app. I have found tons of documents but, however, it take a lot of time and effort... 

The aim

 

 Build a RESTFul API to store and retrive  data. It use  MongoDB and AeroGear to send a notification (when new object is stored).

 

Tools

 

  1.   openshift, as hosting platform.
  2.    Python + Flask, to build the RESTFul API. 
  3.    AeroGear used to push notification. 

Notification 

 

The notifications are send with AeroGear. So I have used one of the openshift gears to hosting this service.  I also  have built an Android App to receive the notification (using Android SDK, AeroGear library for Android and the Google Api to manage  push notifications).

RESTFull API 

 

To build the Flask app I have followed this tutorial and add some code.
This is the required library, in my setup.py:

      install_requires=['Flask==0.10.1', 'MarkupSafe' ,
                        'Flask-SQLAlchemy==0.16','Flask-HTTPAuth==2.2.0',
                        'jsonschema>= 2.3.0','requests'],
Then, as  simple auth method (the same in the tutorial):
@auth.get_password
def get_password(username):
  if username == "myUserName":
    return "myPwd"
  return None

@auth.error_handler
def unauthorized():
  return make_response(jsonify({"error": "Unauthorized access"}), 401)
To initialize MongoDB it's necessary to use the environment variables (OPENSHIFT_MONGODB_DB_URL,OPENSHIFT_APP_NAME), as in the getData  method:

@app.route("/api/getMyData/", methods=['GET'])
def getData():  
  conn = pymongo.Connection(os.environ['OPENSHIFT_MONGODB_DB_URL'])
  db = conn[os.environ['OPENSHIFT_APP_NAME']]
  result = db.mydata.find()
  return str(json.dumps({'results':list(result)},default=json_util.default))
The last, and more complicated method is:
@app.route("/api/setMyData/", methods=['POST'])
@auth.login_required
def createData(): 
  myJson = request.json
  with open(os.path.join(os.path.dirname(__file__),"myData.json"),'r') as f:
    schema = f.read()
  try:
    jsonschema.validate(myJson, json.loads(schema))
  except jsonschema.ValidationError as e:
    return "problem with json schema"    
  conn = pymongo.Connection(os.environ['OPENSHIFT_MONGODB_DB_URL'])
  db = conn[os.environ['OPENSHIFT_APP_NAME']]
  mydata = db['mydata']
  mydata.insert(myJson)
  headers={"Accept":u"application/json", "Content-type": u"application/json"}
  url="https://myOpenshift/ag-push/rest/sender/"
  resp = requests.post(url, data=json.dumps({
    "message": {
    "alert": "My first notification!",
    "action-category": "cat1",
    "sound": "default",
    "badge": 2,
    "content-available": true,
    "user-data": {
      "key": "value",
      "key2": "other value"
 },
    "simple-push": "version=1.2"
      },
    "config": {
      "ttl": 3600
      }}), auth=requests.auth.HTTPBasicAuth("theAeroGearID", "AeroGearAuth"),headers=headers)
  return str(resp)
This method required some extra imports: jsonschema, requests, json.

 At line 4 i get the json from a curl rerquest :
curl -i -H "Content-Type: application/json" -X POST 
--data '{"idStation": 2,"name": "test","lat": 43,
"lng": 11,"z": 200,"stationOwner": "daniele","lastUpdate": "today"}'
 http://myopenschifturl/api/setMyData -u myUserName:myPwd
From line 5 to 8, my json schema is read (from myData.json file) and is verified if  actual json has this schema. To read the file it's necessary to use the variable os.path.dirname(__file__) as the root.

Then if the json is valid, it's stored into the database.  My app manages geographic data, so to use these it's necessary to modify the json, and put latitude and longitude in a loc variable. In order to use it in geographical operation  enable it with   db.points.create_index([('loc', GEO2D)]).

 The last step is  send notification to Aerogear. It's mandatory put the data into json.dumps() to avoid trouble to encoding. theAeroGearID and AeroGearAuth are available in the AeroGear dashboard into application. In the last line I have put the whole signature identifier for HTTPBasicAuth because Flask has it's own HTTPBasicAuth.

Monday, 13 October 2014

GeoGig: an useful tool to work with geographic data

In the past I was looking for a versioning tool for spatial data, but without success... Recently I have found this geogig project, which seems to be what really is suiting my needs. It's  a simple tool for all the guys that have familiarity with git  and versioning and  provides a plugin for geoserver.

I have tried only the basic operations with a shapefile (import, export, add, commit...) and I'm really enthusiastic about how it works. Indeed  my purpose for the near future is to use it in a real project and to better test the features available.