Tuesday, July 10, 2018

Playing with NodeJS and MongoDB Part 2

Playing with NodeJS and MongoDB Part 2



In the part 1 we created the data storage for user data, the user crawler and the mini Node server. Today Ill show you the map that fetches the data, lookup and saves the location and finally show them on the map.


First we write the request handler for listing users. In our existing onRequestReceived() callback lets add a new router option:

 switch (parsed_url.pathname) {
 ...
 case /list:
 doMapDataResponse(response, parsed_url);
 break;
 ...
 }


And the callback:

function doMapDataResponse(response, parsed_url) {
 collection.find(function(err, cursor){
 response.writeHead(200, {Content-Type: script/javascript});
 var items = [];

 cursor.each(function(err, item){
 if (item) {
 items.push(item);
 }
 else {
 response.write(JSON.stringify(items));
 response.end();
 }
 });
 });
}


It uses the same collection of the open db instance, fetches all the stored records and makes a JSON output. Later that will be called from the map page. I think its more re-usable than generating it into the page content.

The other server handler we need is when we got the latitude and longitude coordinates for a location and we update the record with it - so next time we wont hammer the geolocation server. Its as simple as receiving a single record from the client and updating in the db. The router item is:

 switch (parsed_url.pathname) {
 ...
 case /update:
 doUpdate(response, parsed_url);
 break;
 ...
 }


Using the callback:

function doUpdate(response, parsed_url) {
 var item = collection.findOne({username: parsed_url.query.username}, function(err, item) {
 response.writeHead(200, {Content-Type: text/plain});

 if (!item) {
 response.end(Item has not been found);
 return;
 }

 item.lng = parsed_url.query.lng;
 item.lat = parsed_url.query.lat;
 collection.save(item);

 response.end(Data has been updated: + item._id);
 });
}


There we lookup the item - if it exists we update, or die in trauma otherwise.

Now we can start creating a map page. Its a bit tricky. We are using the NodeJS server for pretty much everything here. Because of the same origin policy (yet again bastard) we need to serve the page from the same domain and protocol. I vamped up a quick file server in Node to do that. It requires a new router item:

 switch (parsed_url.pathname) {
 ...
 case /map:
 var fs = require(fs);
 var page = fs.readFileSync(./okcmap.map.html);
 response.writeHead(200, {Content-Type: text/html});
 response.end(page);
 break;
 ...
 }


It reads the file and toss it to the broadband. Thats all we need. Lets make the page then.

<!DOCTYPE html>
<html>
 <head>
 <meta name="viewport" content="initial-scale=1.0, user-scalable=no">
 <meta charset="utf-8">
 <script src="https://maps.googleapis.com/maps/api/js?key=YOUR_GOOGLE_MAPS_API_KEY&sensor=false"></script>
 <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.8.1/jquery.min.js"></script>
 <script></script>
 <style>
 html, body, #map_canvas {
 margin: 0;
 padding: 0;
 height: 100%;
 }
 </style>
 </head>
 <body>
 <div id="map_canvas"></div>
 </body>
</html>


Wireframe is almost by the book - we have the Google Maps API and some jQuery support. The style is a little trick we want to apply - it makes possible to stretch the map out to the full screen estate.

In the header script we make two global variables for the map object and the geocoder service object:

 var geocoder;
 var map;


Then we can initialize the map when the DOM is ready:

 jQuery(function(){
 geocoder = new google.maps.Geocoder();

 var latlng = new google.maps.LatLng(30, -30);
 var mapOptions = {
 zoom: 3,
 center: latlng,
 mapTypeId: google.maps.MapTypeId.ROADMAP
 }

 map = new google.maps.Map(document.getElementById(map_canvas), mapOptions);
 });


Now we have a nice empty map. We can call our JSON resource though Ajax:

 jQuery(function(){

 ...

 jQuery.ajax({
 type: GET,
 url: http://localhost:8888/list,
 dataType: json,
 success: processMapResult,
 error: function(jqXHR, textStatus, errorThrown) {
 // Panic.
 }
 });
 });


Processing the result list splits to two branches - if there is an existing coordinate then we just create a marker and put it out. When its missing we need to call the geolocation service, save the result on the server and then we can create a marker for the map. First case is the easier:

 function processMapResult(response) {
 for (var idx in response) {
 var user = response[idx];
 if (user.hasOwnProperty(lng)) {
 var marker = new google.maps.Marker({
 map: map,
 position: new google.maps.LatLng(user[lat], user[lng])
 });
 }
 else {
 // Missing coords.
 }
 }
 }


As explained the missing coordinate case will do a location lookup. Lets make a handy callback function for that:

 function lookupAddress(address, callback) {
 geocoder.geocode({address: address}, callback);
 }


This has to be asynchronous, so we will make sure that the iterated user data is saved in a scope. In the else branch lets create this scope and the actions:

 (function(user){
 lookupAddress(user.location, function(results, status) {
 if (status == google.maps.GeocoderStatus.OK) {
 jQuery.ajax({
 type: GET,
 url: http://localhost:8888/update?username= + user.username + &lng= + results[0].geometry.location.lng() + &lat= + results[0].geometry.location.lat()
 });

 var marker = new google.maps.Marker({
 map: map,
 position: results[0].geometry.location
 });
 }
 });
 })(user);


Its not the most elegant way to do, but it does the job. And basically thats all. You will see something like this:



The original code has some extras, such as basic error handling. You can find the source in my OkCMap GitHub repo.

If you would like to try it here you are the necessary steps:
  • install NodeJS, MongoDB and the its NodeJS connector
  • get the source
  • run the mongo server: # ./MONGO_BIN_PATH/mongod
  • run the node server file: # node okcmap.server.js
  • add the bookmarklet* to your bookmarks toolbar:
  • go to the okcupid listing page
  • hit the bookmarklet (debug console should show the requests to node)
  • open the map: localhost:8888/map


* the bookmarklet:
javascript:var _=getElementsByClassName;var $=innerHTML;var u=document[_](match_row);for(var i=0;i<u.length;i++){var l=encodeURIComponent(u[i][_](location)[0][$]);var m=parseInt(u[i][_](match)[0][_](percentage)[0][$]);var n=u[i][_](username)[0][$];var url=http://localhost:8888/save?location=+l+&match=+m+&username=+n;console.log(url);var xmlHttp=new XMLHttpRequest;xmlHttp.open(GET,url,true);xmlHttp.send(null)};void(0);


---

Im aware that this is not the best way to solve the problem. I might missed modules in NodeJS to solve sub-problems. If you know a better way, please, share.

Peter

go to link download
Posted by at