Wednesday 23 January 2019

Read magnetic strip card data in Android

Recently I have developed an app with the purpose to read data from an USB card reader. The data to be read are stored on a plastic card organized in this way:
  1.  a start tag;
  2. a fixed field (length of 17 char);
  3. two other fields separated by a white space with no size declarad but always present;
  4. an end tag. 
The external USB card reader is something similar to an external keyboard so it's possible to override the appropriate method in the  Activity to handle every single character read from the reader.
The Activity manages the card data character by character so  firstly I had to create a UsbDecoder class to manage the data flow and to build my output values. Lastly. I had to implemented the onKeyDown to intercept the data from the reader:

 @Override
    public boolean onKeyDown(int keyCode, KeyEvent event) {
        char pressedKey;
        if (event.getKeyCode() == KeyEvent.KEYCODE_SPACE || event.getKeyCode() == KeyEvent.KEYCODE_TAB) {
            pressedKey = Character.SPACE_SEPARATOR;
        } else {

            pressedKey = (char) event.getUnicodeChar();
        }
        if (usbDecoder.addChar(pressedKey)) {
          #do somethings, data are available
        }

        return true;
    }

Unfortunately, the sequence of characters intercepted by this method had some issues as, for example, the fact that the white space was not passed to this method. After several attempts I found a workaround due to the fact that a space charachter is passed to onKeyUp method:


    @Override
    public boolean onKeyUp(int keyCode, KeyEvent event) {
        char pressedKey;
        if (event.getKeyCode() == KeyEvent.KEYCODE_SPACE || event.getKeyCode() == KeyEvent.KEYCODE_TAB) {
            pressedKey = Character.SPACE_SEPARATOR;
            usbDecoder.addChar(pressedKey);
        }

        return true;
    } 
 

Saturday 11 February 2017

RESTFull GEOJson with Swagger and Slim

I often find myself having to write RESTFull api for small project.  So I have search and tried several tools. I finally choose to use Swagger to write the documentation and Slim micro framework to implements it.

The first step is writing the yalm file in the Swagger editor (there is a nice tutorial at Api Handyman blog). With the yalm specification file it's simple to build an html documentation. Swagger have two pre build schema in html (html and html). You can export it, from Swagger Editor with Generate Client button (where you can export other languages client code).

 To create the Slim skeleton, simply click on the Generate server button and choose Slim. Now you have two zip file that contains documentation and the skeleton code.

Usually my project folder is composed by three subfolders: app (with the api code), api_doc (with the documentation generated by Swagger) and then an db_doc (the database documentation with diagram).

To provide a geographic information the request retrive or send a GEOJson, so I have write a  yalm gist as example.

To store the data I use PostgresSQL/PostGIS, using this query to retrive the data:
 
 SELECT row_to_json(featuresCollection)
          FROM ( SELECT 'FeatureCollection' As type,
          array_to_json(array_agg(features)) As features
          FROM (SELECT 'Feature' As type
         ,ST_AsGeoJSON(data.the_geom, 4)::json As geometry
         , row_to_json((SELECT prop FROM (SELECT property1, property2.....) As prop         )) As properties
         FROM myTable As data ) As features ) As featuresCollectionc;

Saturday 14 January 2017

A leaflet template for ionic

Leaflet is a very useful javascript library to insert map into an app. It has lots of extensions, which simplify it's use. I use it mainly in ionic's app, so I have created a repo on GitHub as a start point. It's extends the side menu template with leaflet.

The key points are:
  1. Get a GeoJson through an api call (markersApi.js)
  2. Create a map (with OSM as baselayer).
  3. Add the GeoJson to the map (using the marker-cluster library).
  4. Add a marker on tap.

Add the map to the html

To add leaflet map in ionic (and angular) there is a directive to simplify it: angular-leaflet-directive (or you can use ui-leaflet). In the html you can just put:

<leaflet data-tap-disabled="true" defaults="map.defaults" id="map" layers="map.layers" lf-center="map.center"></leaflet>


And after you have incluse in the app's model dependencies, the angular-leaflet directive, and have defined an object in the controller as $scope.map which contains the center point when load the map, the layer to load and some optional parameter you can see the map.
 
//set the property of the map
$scope.map = {
             center: {
                lat: 46,
                lng: 11,
                zoom: 8
             },
             defaults: {
                zoomControl: false,
                attributionControl: true
             },
             layers: {
                baselayers: {
                    xyz: {
                        name: 'OpenStreetMap (XYZ)',
                        url: 'http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png',
                        type: 'xyz',
                        zoomControl: false,
                        layerOptions: {
                            attribution: '© OpenStreetMap contributors'
                        }
                    }
                }
            }
        };

You also can center the map to the actual position:
 
var options = {enableHighAccuracy: false,timeout: 20000};

//center the map on the actual position.

var setCurrentPosition = function () {
   $ionicLoading.show({
   template: '',
   duration: 7000
   });
   $cordovaGeolocation.getCurrentPosition(options).then(function (position) {
      if (position) {
         $scope.map.center.lat = position.coords.latitude;
         $scope.map.center.lng.longitude = position.coords.longitude;
         } else {}
       }).catch(function (err) {}).finally(function () {
         $ionicLoading.hide();
       });
};

Add markers to the marker

The markers are in a GeoJson format, so they are among throughout the L.GeoJSON layer:
 
//create a marker cluster layer
var markers = L.markerClusterGroup();

/*
*When the view enter:
* get the map.
* get the markers form the api.
* create the geojson layer
* add the geojson layer to the marker cluster layer
* add the marker cluster layer to the map
*
*/
$scope.$on("$ionicView.enter", function (scopes, states) {
   leafletData.getMap("map").then(function (map) {
   map.invalidateSize();
   markersApi.getPoints().then(function (resp) {
      var geojson = new L.GeoJSON(resp, {
          
         onEachFeature: function (feature, layer) {
            //open a ionic modal on click 
            layer.on('click', function (e) {
            $scope.map.center.lat = feature.geometry.coordinates[1];
            $scope.map.center.lng = feature.geometry.coordinates[0];
            $scope.feature = feature;
            $scope.openLayersModal();
            });
           }
      });
   markers.addLayer(geojson);
   map.addLayer(markers);
}, function (err) {
      console.log('ERROR', err.status);
});
});

Add a marker on tap

To add a marker on tap,  you can add this code:

var setMarker = function (latlng) {
   if (!newMarker) {
     newMarker = new L.marker(latlng).addTo(map);
   } else {
      newMarker.setLatLng(latlng).update();
   }
};
var newMarker = null;
map.on('click', function (e) {
setMarker(e.latlng);
});



the repo is available on GitHub at this link.

NOTE: angular-leaflet use leaflet at version 0.7.7 so the marker-cluster is the old version as you can see in the bower.json file.

Monday 2 January 2017

My notes on JavaScript

I started study JavaScript about one years ago, I studied the basic rule and then, due to my work, I moved quickly to use some framework (Ionic and Angular) so I omitted to study it in depth and also  the best practise... Because of this bad start, this December, I re-started to study and I'm writing some notes (mainly on different things than other languages).

Variables

  • keyword const, to declare a constant.
  • Number:
    • Number.NEGATIVE_INFINITY<Number.MIN_VALUE<Number.MAX_VALUE<Number.POSITIVE_INFINITY.
    • Math is an "object" with the most common mathematic functions.
    • Some useful method of Number: parseInt(), parseFloat(), isNaN(), isInfinite().
  • Multiline string, useful to include variable values into the string:
        var a=2;
        var b=2;
        var myString=`this is
                      my value ${a+b} `;

The if statement

  • To test if an array is empty, simply use if(myArray.length) not if(myArray.length>0).
  • To test if a string is void use if(myString).
  • Test only if an object is null if(myObj===null) not if it's null and undefined.

The scope

The scope is a "nested chain": the variable definition is searched in the nested scope, if there isn't then it's search in the upper scope.

Functions

The functions parameters are stored in an object (arguments), similar to an array, so it's possible to retrive parameters with: arguments[i].

Collections

  • Array
    • it can stores several type.
    • it's a map, the key is the index. The length() method return the last index plus 1, so 
          var a=2;
          var myArray=[];
          myArray[25]="myLastObject";
          console.log(myArray.length()); //print 26
      
    • Usefull methods: foreach, concat, join, pop, push, shift, unshift, reverse, sort.
  • Maps, only in ES6.
  • Sets, only in ES6.

Tools

  • bower, package manager.
  • npm, pakage manager.
  • javascriptlint
  • gulp, builder, it's possible run several task (lint, minify resources..).

Saturday 16 July 2016

Notes about SQLite

I'm using SQLite in several project, mainly in mobile app (Android SDK or with Cordova plugin).
SQLite is the best solution to store data locally, but when using it?  Maybe it's better to understand when does't use it:

IF data are detached from the app OR there are lots of concurrent writers OR Big Data

THEN t's better build a server side services (client-server)

OTHERWISE  SQLite


Main Properties:

  1. self-contained
  2. serverless
  3. zero-configuration
  4. transactional: Atomic,Consistent,Isolated, Durable (ACID)

SQLite han't all feature that usually a server side db (as MySQL) have:

  • Right and full outer join.
  • ALTER TABLE isn't completely supported.
  • There isn't Date and Time DataType but it's possible to store data as a REAL (Julian day number), INTEGER (Unix time) or TEXT (ISO8601) and extract it by specific function. I prefer store date as text so for example the today's date is strftime("%s",'now') or use in a query as "strftime('%Y-%m-%d',DATA_RILIEVO) DESC" where DATA_RILIEVO is the columns name.
  • If an INTEGER column is a primary key then it's an alias of the ROWID column. ROWID set is value to first available integer, otherwise it's possible to use AUTOINCREMENT.
  • I have noticed that multiple primary key isn't always supported.


reference

site SQLite



Thursday 31 December 2015

Chart.js 2.0 in Ionic

There are lots JS data chart libraries... it's hard to choose one that's right for you. I started using chartjs because it's responsiveness, it has good support for users interactions and there is an angular directives that support it: angular-chart.js. Angulars directive support chart.js 1.0, after some attempts, I have had some issues:
  1. Time series aren't supported.
  2. It isn't simple to manage missing value.
Then I moved to chart.js 2.0, which was under development. Now, there is a beta version of the library.

So, as first step I have installed it with bower (at https://github.com/nnnick/Chart.js.git#v2.0-dev), and then I try to put it into a Ionic project.
I have put the chart inside a slider:

    <ion-nav-buttons side="left">
        <button class="button button-icon button-clear ion-navicon" menu-toggle="left">
        </button>
    </ion-nav-buttons>

    <ion-content>
        <ion-refresher on-refresh="doRefresh()" pulling-text="Pull to refresh...">
        </ion-refresher>

        <ion-slide-box animation="slide-left-right-ios7" show-pager="true">
            <ion-slide>
                <div class="list card">
    <div class="item">
      First series
                  </div>
    <div class="item item-body">
      <canvas class="chart" height="300" id="firstPlot"></canvas>
    </div>
       </div>
   </ion-slide>
        <ion-slide>
   <div class="list card">
     <div class="item">
  Second
            </div>
     <div class="item item-body">
       <canvas class="chart" height="300" id="secondPlot"></canvas>
            </div>
 </div>
      </ion-slide>
   </ion-slide-box>
  </ion-content>
 </ion-view>
And the controller code:
     
angular.module('myModule', [])controller('chartCtrl', function ($scope, $ionicSlideBoxDelegate,
 $stateParams, $sce , getMyDataServices) {
    Chart.defaults.global.pointHitDetectionRadius = 1;
    Chart.defaults.global.datasetFill = false;
    /*dataToPlot contains 2 series chart1 (which is an array of dimension 3xn => [[],[],[]])
    / and chart2 which is an array 1xn
    / dataToPlot.data contains the labels for x axis.
    */
    var dataToPlot=getMyDataServices.getDataToPlot();
    var chart1=null;
    var chart2=null;
    //if I want to plot an interval.
    var l = dataToPlot.data.length - 1;
    var start = 0;
    //if I want to formatting the data
    var labels = dataToPlot.data.map(function (d) {
                return d;
            });
    chart1 = createPlot(chart1, {series: [dataToPlot.chart1], serieLabel: ["Plot1 "],
                                labels: labels, 
                                colors:['rgb(0, 0,255)','rgb(0, 0, 0)', 'rgb(255, 0, 0)']},
                                start, end, "#plot1a", "[-]");
    chart2 = createPlot(chart2, {series: [dataToPlot.chart2], serieLabel: ["Plot2 "],
                        labels: labels, colors: ['rgb(0, 0, 255)']},
                         start, end, "#plot2", "[-]");

    //update
    $scope.$broadcast('scroll.refreshComplete');
    $ionicSlideBoxDelegate.update();
    $scope.$broadcast('scroll.refreshComplete');

    $scope.doRefresh = function () {
        getData();
        $ionicSlideBoxDelegate.update();
    };
});


var opt = {
    tooltips: {
        mode: 'label'
    },
    hover: {
        mode: 'label'
    },
    scales: {
        xAxes: [{
                display: true,
                scaleLabel: {
                    show: true
                }
            }],
        yAxes: [{
                display: true,
                scaleLabel: {
                    show: true,
                    labelString: "void"
                }
            }]
    }, elements: {
        line: {
            borderCapStyle: 'round',
            borderWidth: 1.0,
            tension: 0.1
        },
        point: {
            backgroundColor: Chart.defaults.global.defaultColor,
            borderWidth: 1,
            borderColor: 'rgb(0,0,0)',
            hitRadius: 10,
            hoverRadius: 5,
            hoverBorderWidth: 1,
            radius: 2
        }
    }
};



var createPlot = function (plot, dataToPlot, start, end, id, scaleLabel) {
    opt.scales.yAxes[0].scaleLabel.labelString = scaleLabel;
    var dataset = [];
    if (!!dataToPlot.series) {
        for (var i = 0; i < dataToPlot.series.length; i++) {
            if (!!dataToPlot.series[i]) {
                dataset.push({
                    label: dataToPlot.serieLabel[i],
                    data: dataToPlot.series[i].slice(start, end).map(function (x) {
                         //parse the value to int, only for me                   
                          return parseInt(x, 10);
                    }),
                    fill: false,
                    borderColor: dataToPlot.colors[i]
                });

            }
        }
        var config = {
            type: 'line',
            data: {
                labels: dataToPlot.labels.slice(start, l),
                datasets: dataset
            },
            options: opt
        };
        if (plot !== null) {
            plot.destroy();
        }
        var ctx = angular.element(document.querySelector(id));
        plot = new Chart(ctx, config);

        return plot;
    }
};



Friday 10 April 2015

A RESTFul with Openshft, Python + Flask and AeroGear

As a new year's resolution I have  started to learn Python. So I have read an introduction to this programming language. Then as next step I have created a RESTful API app. I have found tons of documents but, however, it take a lot of time and effort... 

The aim

 

 Build a RESTFul API to store and retrive  data. It use  MongoDB and AeroGear to send a notification (when new object is stored).

 

Tools

 

  1.   openshift, as hosting platform.
  2.    Python + Flask, to build the RESTFul API. 
  3.    AeroGear used to push notification. 

Notification 

 

The notifications are send with AeroGear. So I have used one of the openshift gears to hosting this service.  I also  have built an Android App to receive the notification (using Android SDK, AeroGear library for Android and the Google Api to manage  push notifications).

RESTFull API 

 

To build the Flask app I have followed this tutorial and add some code.
This is the required library, in my setup.py:

      install_requires=['Flask==0.10.1', 'MarkupSafe' ,
                        'Flask-SQLAlchemy==0.16','Flask-HTTPAuth==2.2.0',
                        'jsonschema>= 2.3.0','requests'],
Then, as  simple auth method (the same in the tutorial):
@auth.get_password
def get_password(username):
  if username == "myUserName":
    return "myPwd"
  return None

@auth.error_handler
def unauthorized():
  return make_response(jsonify({"error": "Unauthorized access"}), 401)
To initialize MongoDB it's necessary to use the environment variables (OPENSHIFT_MONGODB_DB_URL,OPENSHIFT_APP_NAME), as in the getData  method:

@app.route("/api/getMyData/", methods=['GET'])
def getData():  
  conn = pymongo.Connection(os.environ['OPENSHIFT_MONGODB_DB_URL'])
  db = conn[os.environ['OPENSHIFT_APP_NAME']]
  result = db.mydata.find()
  return str(json.dumps({'results':list(result)},default=json_util.default))
The last, and more complicated method is:
@app.route("/api/setMyData/", methods=['POST'])
@auth.login_required
def createData(): 
  myJson = request.json
  with open(os.path.join(os.path.dirname(__file__),"myData.json"),'r') as f:
    schema = f.read()
  try:
    jsonschema.validate(myJson, json.loads(schema))
  except jsonschema.ValidationError as e:
    return "problem with json schema"    
  conn = pymongo.Connection(os.environ['OPENSHIFT_MONGODB_DB_URL'])
  db = conn[os.environ['OPENSHIFT_APP_NAME']]
  mydata = db['mydata']
  mydata.insert(myJson)
  headers={"Accept":u"application/json", "Content-type": u"application/json"}
  url="https://myOpenshift/ag-push/rest/sender/"
  resp = requests.post(url, data=json.dumps({
    "message": {
    "alert": "My first notification!",
    "action-category": "cat1",
    "sound": "default",
    "badge": 2,
    "content-available": true,
    "user-data": {
      "key": "value",
      "key2": "other value"
 },
    "simple-push": "version=1.2"
      },
    "config": {
      "ttl": 3600
      }}), auth=requests.auth.HTTPBasicAuth("theAeroGearID", "AeroGearAuth"),headers=headers)
  return str(resp)
This method required some extra imports: jsonschema, requests, json.

 At line 4 i get the json from a curl rerquest :
curl -i -H "Content-Type: application/json" -X POST 
--data '{"idStation": 2,"name": "test","lat": 43,
"lng": 11,"z": 200,"stationOwner": "daniele","lastUpdate": "today"}'
 http://myopenschifturl/api/setMyData -u myUserName:myPwd
From line 5 to 8, my json schema is read (from myData.json file) and is verified if  actual json has this schema. To read the file it's necessary to use the variable os.path.dirname(__file__) as the root.

Then if the json is valid, it's stored into the database.  My app manages geographic data, so to use these it's necessary to modify the json, and put latitude and longitude in a loc variable. In order to use it in geographical operation  enable it with   db.points.create_index([('loc', GEO2D)]).

 The last step is  send notification to Aerogear. It's mandatory put the data into json.dumps() to avoid trouble to encoding. theAeroGearID and AeroGearAuth are available in the AeroGear dashboard into application. In the last line I have put the whole signature identifier for HTTPBasicAuth because Flask has it's own HTTPBasicAuth.