Saturday, March 30, 2013

Is this volume offline?

Apologies

First of all, let me apologize for this silence since June 2012. In August 2012, my wife and I decided to go back in France with our daughter. Moving from Africa to Europe was not as seamless as we thought it would be. But everything is now back in place and my french company is now running since March, 1st.

First project!

For my first project as a French company, one of the constraints is: cross-platform. Mac OS x and Windows for sure, and at some point it will have to run on Linux. Given that, the IDE choice was simple: Real Studio. This application will have to watch for remote volumes and if it finds a specific folder and file hierarchy, then it will have to copy some of the files to another specific file structure that resides on local drives. As the remote volumes are not always on line, the application needs to know when a drive is mounted or unmounted.

I searched for a ready-to-use cross-platform solution to accomplish that. I also ask the NUG maillist and I found the Windows Functionality Suite or MacOsLib that are avalaible for free but that come with poor documentation, if any. There is the MBS plugins, documented, but still not really cross-platform. And I'm always lost in its documentation. Then I thought about something Charles Yeomans once wrote on the NUG. I don't remember the exact words but he said that there is nothing a plugin can do that can't be done in Real Studio with or without Declare. So I decided to write my own solution in pure REALbasic.

In pure REALbasic?

As a prototype, I created a Thread subclass that I named zdVolumeWatcher and add two event definitions.

Event VolumeMounted(inNativePath As String, inFolderItem As FolderItem)
Event VolumeRemoved(inNativePath As String)

Then to remember what was the configuration at the last check I decided to use a dictionary and to make it private as it's to be used by the class only.

Private pVolumeList As Dictionary

The key will be the shell path of the volume, as on MacOS X we can be sure that it's unique even if there are volumes with the same name. The value will be the FolderItem returned by the Volume() function. The dictionary will be initialized in the class constructor.

Sub Constructor()
  // Create the volume list dictionary
  Me.pVolumeList = New Dictionary
End Sub

Let's go for the Run() event code. After declaring the variables the code will use, it enter an endless loop. Not quite endless indeed but we'll discuss that later. The first thing the code is doing is to collect all the volumes returned by the Volume() function, store them in an array of FolderItem and also store the volume's shell path in an array of String.

Then the code searches for new volumes. It checks that the shell path of each collected element already exist in the pVolumesList() dictionary. If not, the volume's FolderItem is added to the dictionary using its shell path as key and the VolumeMounted event is raised.

Last step, the code searches for removed volumes. This time it uses the pVolumeList dictionary and try to find their shell path in the thePaths() array. If it failed to find it, this path is removed from the dictionary and the VolumeRemoved event is raised

Sub Run()
  //-- Watch the attached volumes

  #pragma DisableBackgroundTasks

  // Setup the config
  Dim theVolumes(), theItem As FolderItem
  Dim thePaths(), thePath, theKey As String
  Dim theLastIndex, theIndex As Integer

  // Let's go for an endless loop
  Do
 // --- Collect the volumes ---

 // Redim the arrays
 theLastIndex = VolumeCount - 1
 Redim theVolumes( theLastIndex )
 redim thePaths( theLastIndex )

 // Fill the volume arrays from the
 For theIndex = 0 to theLastIndex

   theItem = Volume( theIndex )
   theVolumes( theIndex ) = theItem
   thePaths( theIndex ) = theItem.ShellPath

 Next

 // --- Check for New Volumes ---

 theLastIndex = thePaths.Ubound
 For theIndex = 0 to theLastIndex

   // Retrieve the volume path
   thePath = thePaths( theIndex )

   // Was it there last time we checked?
   If Not Me.pVolumeList.HasKey( thePath ) Then

  // No, this is a new volume, add it to our volume list...
  Me.pVolumeList.Value( thePath ) = theVolumes( theIndex )

  // ... And tell the world "Habemus volumus"
  RaiseEvent VolumeMounted( thePath, theVolumes( theIndex ) )

   End If
 Next

 // --- Check for removed volumes ---

 For theIndex = 0 to Me.pVolumeList.Count - 1

   // Retrieve this known volume Key ( Shell Path )
   theKey = Me.pVolumeList.Key( theIndex )

   // Is it still there?
   If thePaths.IndexOf( theKey ) < 0 Then

  // No, this volume has been removed
  // Tell the world...
  RaiseEvent VolumeRemoved( theKey )
  // ... and renove it from our list
  Me.pVolumeList.Remove( theKey )

   End If

 Next

 // Let's get some sleep
 Me.Sleep( 1000 ) // 1000 msecs = 1 sec.

 // Loop until being told to stop
  Loop Until Me.pStopFlag

  // Some housekeeping never hurt
  Me.pVolumeList = Nil
  theItem = Nil
  Redim theVolumes( -1 )
  Redim thePaths( -1 )
  // We're done here.
End Sub

You may have noticed the Loop Until Me.pStopFlag line. pStopFlag is a private property that is set in a method called StopNicely. If you want to stop the thread nicely, just call this method. It will stop it at the end of a checking loop, not in the middle of it.

Sub StopNicely()
  //-- Sets the stop flag
  Me.pStopFlag = True
End Sub

So, the thread checks the volumes configuration each second or so. Be aware that if a volume is connected and disconnected while the thread is sleeping, nothing will happen. You still can set the thread not to sleep, but it will just consume CPU power.

Using it for real

To use zdVolumeWatcher, you can subclass it to directly access the Event or use the AddHandler keyword to delegate the Event handling to one of your own methods.

Thread vs. Timer to update the GUI

Well, in the project I will use this class to send notifications to a 'NotificationCenter' class of my own, also written in pure REALbasic. This class will spread the notification in the others elements of the application. What happens if you just want to use zdVolumeWatcher to directly update the user interface and don't want to deal by the Threads can't interact directly with GUI thing?

Very simple to do... Duplicate the zdVolumeWatcher class and rename the copy to zdVolumeWatcherTimer. Now, cut & copy the code from the Run() event to the action event. A this point, the Run() event should be gone. In the Action() event, delete the line with Do at the beginning of the loop and the all the lines at the end of the loop and listed below.

 // Let's get some sleep
 Me.Sleep( 1000 ) // 1000 msecs = 1 sec.

 // Loop until being told to stop
  Loop Until Me.pStopFlag

  // Some housekeeping never hurt
  Me.pVolumeList = Nil
  theItem = Nil
  Redim theVolumes( -1 )
  Redim thePaths( -1 )

You can also remove the pStopFlag property and the StopNicely method. Now you can drop the zdVolumeWatcherTimer on a window layout and fill the VolumeMounted and VolumeRemoved events to intract with the GUI without any problem.

These classes are not perfect, and their code could be improved for sure. But the project is a young. If you see some improvements, feel free to post a comment. I'd be glad to update this post with your suggestions.

Monday, June 18, 2012

Upside Down Arrays & A Common Ancestor

Plenty Of Useful Built-In Methods

REALbasic, the language, provides us with some interesting methods to manipulate arrays: Array, Join, Split, Append, IndexOf, Insert, Pop, Redim, Remove, Shuffle, Sort, SortWith, etc... You'll find an exhaustive list in the Language Array category in Real Software's Wiki.
What you may notice is the lack of a method to reverse the order of the elements in the array. But this kind of manipulation shouldn't be hard to code in REALbasic. Let's give it a try!

Tuesday, May 1, 2012

More about how to use GPSKit for Real Studio

While I was completing the GPSKit documentation, I felt like I may share the some bits of it on this blog. I hope you'll find them useful and feel free to leave a comment if you wish to.
Just as a reminder, GPSKit 1.0beta2 is downloadable here.

The design of GPSKit.

GPSKit is based on a separation of tasks design. They are 4 layers of them:

  1. The first layer is the hardware connection. It gets the data from the GPS device and spit out valid chunks of data 'tokenized' according to the protocol specifications. It's the zdGPSDataProvider interface.
  2. The second layer is the data validator and parser. It receives the tokens from the zdGPSDataProvider, check their validity and turn each of them into an object that represents the GPS raw data in a more human understandable way. It's the zdGPSDataListener interface .
  3. The third layer is a GPS fix factory. It uses the objects instances provided by the zdGPSDataListener to create a zdGPSFix instance and make it available to the entire application. it's the zdGPSFixProvider interface.
  4. As in any I/O protocol, errors may happen, so there is a fourth layer that will handle these errors and perform the actions needed in such case. It's the zdGPSErrorHandler interface.

In order to know where to send the data, the zdGPSDataListener needs to say hello, 'to register' itself to the zdGPSDataProvider. As the zdGPSFixProvider is meant to be implemented within the same class as the zdGPSDataListener, there is no need to register it. However, it isn't mandatory for the zdGPSDataListener to implement the zdGPSFixProvider as some protocols may not provide geolocation information.

Currently, only the NMEA 00183 protocol via a serial port is available in GPSKit. For this format, the zdGPSDataProvider layer is implemented by the zdNMEASerial class. The zdGPSDataListener and the zdGPSFixProvider are both implemented by the zdNMEAParser. The zdGPSErrorHandler is implemented by the zdGPSDeviceConnection.

Getting started.

If you need to get the geolocation data provided by the GPS device connected to a serial port, you'll have to create a data provider, a data listener that is also a fix provider, register the listener to the provider, store the references to all these class instances and finally open the connection. Fortunately, zdGPSDeviceConnection is here to automate this process. It implements a method with the following syntax:

  Sub BuildSerialConnection( _
        inProtocol As zdGPSDeviceConnection.Protocol, _
        inSerialPort As SerialPort)

Where inProtocol is one of the value of the zdGPSDeviceConnection.Protocol ('UserImplemented' and 'NMEA0183' are the only currently available values), and inSerialPort is the SerialPort instance the GPS device is connected to. It will handle the serial settings depending on the protocol specifications (e.g. 4800 bauds, 8N1 for the NMEA 0183).
Note: The 'UserImplemented' protocol is currently used for internal testing purpose only. This may change in a future release.

So to build and open a connection, proceed as following:

  Dim theConnection As New zdGPSDeviceConnection

  theConnection.BuildSerialConnection( _
        zdGPSDeviceConnection.Protocol.NMEA0183, _
        System.SerialPort( 0 ) )

  If theConnection.Open Then
    MsgBox "Connection is open"
  Else
    MsgBow "theConnection failed to open"
  End If

Of course, you'll have to adapt the code depending on the serial port you are using and set your GPS device interface type to 'NMEA 0183'.

If the connection is successful, you can access the

  zdGPSDeviceConnection.NMEAParser.GetLatestGPSFix() As zdGPSFix

to get the geolocation data. Don't forget to test for 'Nil' before using the returned reference.

Something you should be aware of.

The BuildSerialConnection() method will raise a zdGPSKitException if the passed serial port reference is 'Nil' or if the chosen protocol is 'UserImplemented'. In both cases, the Message property content explains, in english, why the exception has been raised.

Why using a more complicated design when a single class handling all the data processing would have been simpler?

While the format of data may be the same (e.g. NMEA 00183), the type of hardware connection may be different depending on the GPS device you're using (e.g. serial port, USB, even TCP/IP). So to avoid duplicate code for each connection type, a multiple layering with interfaces was chosen. Actually, the first reason was to allow the testing of the code by using a file that simulates the input of serial data. A little application, developed with Real Studio, was used to record the data coming from the GPS device along with the timing information to a binary file while driving a car. Back to the IDE, A class implementing the zdGPSDataProvider interface was used to replay the flow of data with the same timing. The application and the replay class are not included in the current release because they really can use some refactoring, but may be in another release...

How GPSKit was developed.

Two Garmin GPS receivers are used to develop GPSKit: A GPS 72 and an eTrex H. A Nokia Navigator 6110 cell phone was also used via Bluetooth as well as some serial data from a Garmin Rhino 130 and a GPS III previously recorded to binary files.

The code was written and tested with Real Studio 2009r4 ( Enterprise edition ) on a MacBook 13". A HP nw9400 laptop and a Acer Aspire One ZG5 notebook are used to test on Windows XP. The Acer notebook is equipped with A 32GB SSD disk and is mostly used to record the GPS serial data when driving a car.

One more thing...

Doing something else than driving when your vehicle is moving is very dangerous. Always park your vehicle safely before using a GPS device, a computer, a cell phone or any other kind of stuff that needs at least one of your hands and draws your attention away from driving or riding.

Friday, April 27, 2012

GPSKit 1.0b2: The first public release for Real Studio

After being away for a while, I'm back again with some hot news: The first public beta release of GPSKit for Real Studio is now available for you to test it. A zip archive containing the demo app and a draft of the the documentation is available here.

The set of classes comes within a demo app and with a sketchy documentation that will be completed along the beta testing period. All  the classes are encrypted and their use is time limited: 10 minutes for a debug build and 5 minutes in a compiled app. An unlocking scheme will be implemented before the final candidate stage. The selling price hasn't been fixed yet.

To open a connection to your GPS device, create a new zdGPSDeviceConnection, pick a protocol from the zdGPSDeviceConnection.Protocol enumeration ( currently limited to NMEA 0183 ) and a reference to the SerialPort your GPS device is connected to. Then call the zdGPSDeviceConnection.BuildSerialConnection() method. You can see an example of how it works in the TestWindow.ConnectButton.Action event.

If you find a bug, a typo or think of a missing feature, feel free to create a report here. A subscription is required.
I'd be glad to hear any thought, critic or advice you may have about GPSKit, so don't hesitate to post a comment below.

The two top items on the GPSKit's to-do list are:
  1. Improve and complete the documentation to make it really useful.
  2. Implement the text output protocol.
They will be the main focus of the GPSKit v1.0b3 release. Stay tuned!

Tuesday, April 17, 2012

After a desert trip...

As I was in the South Moroccan Sahara desert for the last ten days to work with the TV production crew of the "Marathon des Sables" race, I had to temporarily pause the GPSKit development. But I found the opportunity to record some GPS NMEA data stream that should prove useful to test GPSKit in a more "real life" way.
I also got some pictures of our roaming technical setup:

  • 4 cameras using the Panasonic P2 cards to record video.
  • An Avid LANShare video server with 4 TB of storage.
  • An in & out media station
  • three video editing "suits"
  • A Satellite broadcasting station
I will make an entire post about it in a few days.