History
How many ways do you have to get home from work? I have about 8 different possibilities I'm interested in. Going from home to work I have three. There's probably lots of others, some of which may even be as viable as the ones I'm interested in.So which one's the best?
This idea came to me as I was driving home on Friday, November the 4th, 2005. I was trying a new route home that a co-worker had suggested[1]. I looked at my car's clock and noticed that I had left work a little later than usual. I wasn't sure what a good arrival time woud be. As far as I could tell just by guesstimation, the new route was pretty close to the old one. Not worse nor better as far as I could tell.
That naturally got me thinking about ways I could figure out which route's the best. I had just started playing with audio blogging via a voice recording MP3 player[2]. This lead me to thinking that I could record my drive home. While driving home I would yell out stuff like "Now at Island Park bridge!" or "Now at Carling and Moodie!" Reviewing the audio file, I could use the time index for each of those yellings to figure out how long each leg of my journey was, and therefore determine which route is best.
As it turns out, the voice recording MP3 player has a glitch. It's not a UMS device and the software I was using to download it was introducing errors in the form of missing chunks of data[3]. Missing chunks of data is bad when you want to need a complete time index for accurate measurements.
Still, there had to be a way to get an accurate measurement of how long it takes to drive the various parts of the route....
[1] When I first bought my house, I had started out taking a different route home, my initial route. I switched to a new route shortly thereafter because, although it didn't seem significantly shorter, it traded sitting in stop-and-go traffic from the middle of my journey to the beginning (I'd rather suffer through that right away and let the fun afterwards be uninterrupted). My co-worker's suggestion was the third route.
[2] This idea sucked. The audio quality was decent. Content? Horrible.
[3] Either that or the device itself was faulty. Either way, not so good.
Introduction
This project will involve measuring and analysing how long it takes for me to drive to and from work using various different routes, with the goal of determining which route is the best. I'll[1] use GPS to collect the raw data. I'll write little Perl scripts to analyse the data and then plot results using the Google Maps API[2].[1] Yeah, yeah, I know. Proper experiments should use The Scientific Method and the report shall be written in the 3rd person passive voice. This will not be a proper experiment. I may even add (*gasp*) humour!
[2] I was going to use Perl and Gnuplot, but the Google Maps API is so freakin' cool, it's boss, man! (you know it's cool when I have to revert to 60's slang)
Equipment
-
A car
A GPS
A Laptop
A 300W power inverter
A USB<->RS232 cable
The power inverter powers the laptop[4] while it's in the car. The USB<->RS-232 cable is required because certain laptop manufacturers don't see the need to provide their customers with RS232 serial ports, the bastards.
[1] Data points include timestamp, position in lat/long, speed, and a whole whack of other data, some of which I might also find useful
[2] OK, OK, on my couch with a tall glass of smoothie nearby.
[3] D'ya see that? I've gone back to 3rd person passive again! Them schools sure learn ya good!
[4] If ya wanna be picky about it, the car powers the inverter, the inverter powers the laptop's transformer, the transformer recharges the battery, the battery powers the laptop. The kneebone's connected to the hip-bone....
Definition of terms
| Trip | There are two trips in this project: the morning home-to-work trip, and the evening work-to-home trip. |
| Route | A full one-way trip, from start to finish. There are multiple routes per trip. |
| Leg | A part of a route which is different from other routes but shares a startpoint and an endpoint with at least one leg of a different route. It could also be a part of a route that is identical to parts of other routes but that shows variation, e.g. traffic variances or traffic light time variances. |
| Waypoint | The point at which a particular leg starts or stops, or other interesting points along a route. A waypoint found on one route should show up on at least one other route. |
| Perl | An instantiation of the TMTOWTDI class of high-level languages. |
| Google Maps API | An incredibly easy-to-implement method of presenting map-oriented data (like GPS routes, waypoints and information snippets). Drawing routes using little line segments whose colour will indicate the speed at that time[1], making little markers for waypoints which, when you click on them will pop up a window detailing the elapsed trip time to get to this point for a particular route (or set of routes) |
[1] Right now I'm thinking that the scale will be something like red=0km/h, green=50km/h, blue=100km/h and colours will fade between them depending on the speed. For example, 25km/h will be 50% red, 50% green (over 100km/h and we start fading to yellow):
0 . 10 . 20 . 30 . 40 . 50 . 60 . 70 . 80 . 90 . 100 . 110 . 120 . 130 . 140 . 150
Status / Logbook
| TIMESTAMP | DESCRIPTION OF STATUS |
| 20051104-17h45 | Initial conception of project idea. Various refinements over the weekend. |
| 20051107-19h00 | Begin writing project webpage. |
| 20051108-12h30 | USB<->RS-232 cable purchased, because my steenking laptop does not have a built-in RS-232 serial port. |
| 20051108-14h30 | Discovery of Google Maps API via co-worker. Resulting changes in project plan ensue. |
| 20051108-19h00 | Performed test run in Rewired power inverter to use 12VDC cigarette lighter adapter. |
| 20051108-19h46 | Test run starts. I hooked everything up in the car as it will be during data production and went for a test drive. |
| 20051108-20h08 | Test run complete. Return to lab. |
| 20051109-07h45 | Finished initial script that overlays the test route onto a Google Map. This attempt used an array of points in a single GPolyline[1]. It didn't include colouring for speed. This is also the point where I learned that the map crashes when you put more than 500 or 600 points in an array to be used in a GPolyline[2]. |
| 20051109-12h50 | Modified script to use line segments with speed colouring. Learned that Google Maps can handle at least 1200 GPolylines (each with only two points and a colour) but that it's really slow with that many points. REALLY SLOW. Oh, and it uses a lot of bandwidth, too. My test run was only 40 minutes long. My real routes will be up to an hour long, and I'll probably want to plot multiple routes at a time on a single map. |
| 20051109-19h45 | A new version of the script is complete, with variable resolution. I use a formula that effectively looks at changes in bearing and speed[3] to decide how many data points to include. In the graph to the right, I've plotted the percentage of data points used in the map as a function of the selected delta-speed. As you can see, resolution decreases exponentially, and somewhere between 10 and 20 is a good number to use.An interesting thing to note is that even using 0km/h (which should theoretically plot every point where the speed of the vehicle changes in any direction by any amount), we get only 86% of the data points. The remaining 14% are points where the vehicle is stopped. |
| 20051110-06h47 | I recorded my trip to work this morning as a second test run. |
| 20051110-07h30 | Updates to genmap_header.html: when you click the map, it will now tell you the lat/long of the point you clicked, plus the lat/long/zoomlevel of the center of the map. Updates to genmap.sh: it will use the center/zoom that you specify for the initial position of the map.
|
| 20051110-17h20 | I recorded my trip home this morning as a third test run. |
| 20051110-19h00 | I wasted a lot of time trying to figure out the best way to place waypoints. You can see the results in the Trips, routes and waypoints section. |
| 20051111-06h47 | Trip to work this morning makes the fourth test run. Slightly different route than 2nd test run. |
| 20051111-12h30 | Thought a lot about where to place waypoints (including writing a lot of crap on a webpage) but eventually decided it'd be much less time consuming to simply make some guesses as to where the waypoints should be, collect a bunch of data, and then try out different theories against the data and see which works best. |
| 20051111-17h17 | Trip home this afternoon makes the fifth test run. Slightly different route than 3rd test run. |
| 20051111-21h00 | Came up with a flowchart format to express the various routes & waypoints of a trip. |
| 20051112-09h30 | Thought a lot about some viable options: (1) On the Start→Maisonneuve via St. Redempteur, I could turn at either Sacre Coeur (new option) or St. Laurent (current plan). Rejected this option because it's too granular -- I'm confident that the difference won't be more than a minute, and that's too granular to worry about adding a new route for. (2) I could leave Carling to get on the Queensway at March (new option) vs. Queensway at Moodie (current plan). Rejected this option because I can't see this working. When I get on the Queensway at Moodie, I move at 70-90km/h. I doubt I can do better than that on Carling, and the March route is longer. The only way to beat a longer distance is with a higher speed (the sucky thing is I wasted about 5 minutes adding this to the flowchart before looking at the map and rejecting it). (3) I could leave Carling at Holly Acres and get on the Queensway at Bayshore (new option) instead of leaving Carling for Queensway at Moodie (current plan), or even (4) leave Carling at Corkstown for Queensway at Moodie (new option). These options both seem viable -- they'll be slower, but they're also shorter distance-wise. It's a trade of speed for distance, and it might work, so I'll try them and find out. |
| 20051113-17h00 | Created the flowcharts and write-up for the home-work routes. Removed the Holly Acres option from the work-home route because there's no way to get on the Queensway from there. Added a Richmond route. I've gotta stop messing with these routes and get to some actual coding. |
| 20051116-19h00 | Updates to this webpage have become less frequent. Things I've done in the last couple of days: 1) data collection twice a day; 2) wrote some wrapper code to generate PNG images; 3) updated the main datafile parsing code to track elapsed times and look for stops; 4) wrote some scripts that will automatically re-parse all the files to generate their GMap webpages using an optimal resolution. Working on: 1) Waypoint tracking code; 2) the code that automatically parses all the files will also build an automated results webpage. |
| 20051117-19h00 | I re-built the code to use hashes for storing datapoints, and generally cleaned up the code a little bit. There's also some code now that will parse all datafiles and auto-build the Results Page. The only part that isn't automated is the actual collection of screenshots. This is mostly automated, and finishing it is a low priority. |
| 20051120-19h00 | After solving a deep-copy issue with my hashes, I've discovered that I need to add a new category of waypoint. Let's say that I'm going home from work, passing through waypoint Queensway @ Castlefrank going via Terry Fox to the end. There's another choice at that point, which is go via Castlefrank. When the analysis spits out the results, it would be nice if it said "Queensway @ Castlefrank via Terry Fox to Finish" or something like it. So I'll put "decision marker" waypoints along the path of each possible chosen leg. Each waypoint will have a type-flag (the old ones are "wpt", these are "dec", and there could be others) and only waypoints which are "wpt" are used in calculations. |
[1] This is Google-Maps-API-specific techno-speak. If you'd said this to me three days ago I'd'a said, "Whatchu talkin' 'bout, Willis?"
[2] It's unclear to me if it's Javascript or Google Maps which can't handle the array.
[3] ... although it's coded in a quick-and-dirty way which may or may not work well for other modes of transportation (walking, biking) or even other drivers' driving styles. I'm using the classic "Kick out an ugly version 1 as soon as possible" approach to coding.
Procedures
This part fluctuates more than a flag on a windy day, but here's what I've come up with so far.| Data collection | This is pretty easy, really. I get in the car, plug the computer in, turn the GPS on, begin logging to file on the laptop, and drive the car. When I get to where I'm going, I save and close the file, shut the GPS off, unplug the laptop and leave. It's so simple I don't even need to use 3rd person passive voice to describe it! |
| Route following | There will be a set of routes per trip. On each trip where I collect data[1] I will rotate through the routes in an ordered fashion. The idea is to hit all the routes in a trip's set as soon as possible to minimize long-term variance[2]. |
| Data analysis | This is too up-in-the-air to describe in detail. Right now I envision writing a script that will parse the raw data file to produce a map that includes elements like: 1) line segments which connect the dots to form the route, with colouring to indicate speed for each dot; 2) markers to indicate elapsed time at waypoints along the route; 3) other markers to indicate miscellaneous informational tidbits as needed.
I'm thinking that a quick written summary of each route to note specifics (such as route ID number) and extraordinary events (short-term variances, etc.) will probably accompany each map. |
[1] Some trips may be so anomalous that it's not worth collecting data, e.g. Corel centre hockey night work-to-home trips. Then again, on those days I may decide to hang with friends in Ottawa.
Every trip where I do collect data will be followed in an identical fashion, e.g. direct from work to home, no stops for shopping, no passing GO to collect $200.
[2] Variances are changes to aspects of the route which affect the data, like traffic volume, signals, etc. Long term variance are changes, such as construction, bus strikes, changes brought on by weather (e.g. road conditions, increase in car volume due to seasonal biker retirement, etc.) which, because they change over longer periods of time, affect multiple trips.
Short term variances are things like accidents or freak traffic patterns (Corel Centre hockey nights, etc.) which affect only a single trip.
Trips, routes and waypoints
I was actually thinking through which place is the best to put the waypoints (in the diagram -- which you can click to enlarge -- consider putting your waypoints at the green, blue or red arrow, respectively) but there are so many possibilities with different things to consider for each possibility that it was starting to branch out into a very complicated tree.I wasted about four hours weighing merits of different placement strategies before I finally decided it would cost less time to determine the best waypoint strategy by analyzing the data.
| This is the trip that started the whole project. |
v1.0: 10 initial combinations
v1.1: 18 combinations. Added choice of Queensway @ March vs. Queensway @ Moodie.
v1.2: 26 combinations. Decided Queensway @ March will suck. Added Holly Acres and Corkstown options.
v1.3: 26 combinations. Holly Acres is no good, but Richmond is an option.
START POINT:
St. Joseph & St. Raymond, Hull GatineauFINISH POINT:
Glencairn, Kanata OttawaROUTE SEGMENTS:
S-TM-1) START → St. Raymond / Montagne → Tache @ Montagne S-TM-2) START → St. Joseph → Tache → Tache @ Montagne TM-CO) Tache @ Montagne → Champlain → Champlain Bridge @ Ottawa River Parkway S-SM-1) START → St. Joseph → Montclair / St. Redempteur → St. Laurent → St. Laurent @ Maisonneuve S-SM-2) START → Hwy. 5 → Maisonneuve → St. Laurent @ Maisonneuve SM-CO) St. Laurent @ Maisonneuve → Ottawa River Parkway → Champlain Bridge @ Ottawa River Parkway CO-CR) Champlain Bridge @ Ottawa River Parkway → ORP → Carling → Carling @ Richmond CR-QB) Carling @ Richmond → Richmond → Queensway @ Bayshore S-QB) START → Hwy. 5 → King Edward → Mann → Queensway → Queensway @ Bayshore QB-QM) Dude! Just stay on the Queensway! CR-CC) Carling @ Richmond → Carling → Carling @ Corkstown CC-QM-1) Carling @ Corkstown → Carling → Moodie → Queensway @ Moodie CC-QM-2) Carling @ Corkstown → Corkstown → Moodie → Queensway @ Moodie QM-QC) Queensway @ Moodie → Queensway @ Castlefrank QC-WR-1) Queensway @ Castlefrank → Castlefrank → Winchester @ Ricky QC-WR-2) Queensway @ Castlefrank → Queensway → Terry Fox → Winchester @ Ricky WR-F) Winchester @ Ricky → McElroy → Morton → FINISHDISTINCT WAYPOINTS:
ROUTE ROTATION SCHEDULE:
-
King Edward / Castlefrank
King Edward / Terry Fox
St. Joseph / Richmond / Castlefrank
St. Joseph / Richmond / Terry Fox
St. Joseph / Corkstown / Castlefrank
St. Joseph / Corkstown / Terry Fox
St. Joseph / Moodie / Castlefrank
St. Joseph / Moodie / Terry Fox
St. Raymond / Richmond / Castlefrank
St. Raymond / Richmond / Terry Fox
St. Raymond / Corkstown / Castlefrank
St. Raymond / Corkstown / Terry Fox
St. Raymond / Moodie / Castlefrank
St. Raymond / Moodie / Terry Fox
Montclair / Richmond / Castlefrank
Montclair / Richmond / Terry Fox
Montclair / Corkstown / Castlefrank
Montclair / Corkstown / Terry Fox
Montclair / Moodie / Castlefrank
Montclair / Moodie / Terry Fox
Hwy. 5 / Richmond / Castlefrank
Hwy. 5 / Richmond / Terry Fox
Hwy. 5 / Corkstown / Castlefrank
Hwy. 5 / Corkstown / Terry Fox
Hwy. 5 / Moodie / Castlefrank
Hwy. 5 / Moodie / Terry Fox
| Can't do one without the other, eh? |
v1.0: 5 initial combinations
START POINT:
Glencairn, Kanata OttawaFINISH POINT:
St. Joseph & St. Raymond, Hull GatineauROUTE SEGMENTS:
S-WR) START → Morton → McElroy → Winchester @ Ricky WR-TH) Winchester @ Rickey → Winchester → Terry Fox → Terry Fox @ Hazeldean TH-HC) Terry Fox @ Hazeldean → Hazeldean → Hazeldean @ Castlefrank WR-HC) Winchester @ Rickey → Winchester → Castlefrank → Hazeldean @ Castlefrank HC-QMa) Hazeldean @ Castlefrank → Hazeldean → Eagleson / March → Queensway @ March QMa-QMo) Dude! Just stay on the Queensway! HC-QMo) Hazeldean @ Castlefrank → Hazeldean / Robertson → Moodie → Queensway @ Moodie TH-QMa) Terry Fox @ Hazeldean → Terry Fox → Queensway → Queensway @ March QMo-F) Queensway @ Moodie → Queensway → Lees Av. → King Edward → Hwy 5. → FINISHDISTINCT WAYPOINTS:
| START |
Glencairn, |
| WR | Winchester @ Ricky |
| TH |
Terry Fox @ Hazeldean |
| HC |
Hazeldean @ Castlefrank |
| QMa |
Queensway @ March |
| QMo |
Queensway @ Moodie |
| FINISH |
St. Joseph & St. Raymond, |
ROUTE ROTATION SCHEDULE:
-
Terry Fox / TF-Queensway
Terry Fox / Hazeldean / Eagleson
Terry Fox / Hazeldean / Robertson
Castlefrank / Eagleson
Castlefrank / Robertson
Results
| Scripts 'n' stuff |
| ||||||||||||
Click on the map images below to get to the actual Google Maps. Keep in mind that they may take awhile to load, so be patient. Once they load, you can scroll and zoom as with any GMap. If you click on any particular point (which isn't a marker), it will create a marker whose informational window will tell you that point's lat/long as well as the lat/long/zoomlevel of the center of the map. | |||||||||||||
| Test run #1 | The results of my first test run. I drove from my house to Terry Fox to the Queensway eastbound to Moodie, where I turned around and went westbound on the Queensway, getting off at Castlefrank and returning home via a circuitous route.
| ||||||||||||
To-do list
| Todo list | Put todo list on website | Completed (20051110-08h15) |
| Route planning | Update the website's routes section with all the routes and waypoints and stuff. | Completed (20051113-17h00) |
| Usage info | Make genmap.sh and create-googlemap.pl spit out usage information if required cmdline args are not given. | Not done |
| Opacity | Increase the opacity of GPolylines (need to rebuild all maps once done) | Completed (20051114?) |
| HTML style updates | Update HTML header/footer to include tracking snippet and some styles. We'll use the right side of the screen for map title, summary and other information, such as route summary data, waypoint data, other informational tidbits. I envsion each waypoint marker having a letter/colour to identify it, and copies of the markers will appear on the right with information about each beside it. | Partially complete (20051110-12h00) |
| 5min time markers | Write code to put markers every five minutes. The information window should have something like: elapsed time, distance traveled, current posn/speed? Eventually the icon used should be a clock icon. | Rejected; will not do |
| Waypoint markers | Write code to find the the data points in the file which are nearest to a given set of lat/long waypoints. Said code will create a marker which will have an informational window displaying elapsed time, distance and speed. Each waypoint should have its own icon, probably a standard teardrop with different letter/ID and colour. | Not done |
| Stop points | Write code to find stop points and create a marker (eventually this marker will be an octagonal STOP sign) whose informational window will display the elapsed time when stop began, when started moving again (and the delta), elapsed distance. | Partially completed (20051116-08h30) |
| Optimization w/ Javascript | Change the code so that we just put all the points in an array. Write Javascript function which will look at the current map boundaries and draw only the points which lie within. Call this function on intial load and when repositioning or rezooming occurs. Include in the array a number which indicates the zoom level the given point is appropriate for -- so that at full zoom-in, all points are drawn, zoomlevel2, say a resolution setting of 5, zoomlevel6, resolution setting 15, or something like that.. | Not done |
| Waypoint to waypoint detection code | Write code to detect when a track goes from any given waypoint to any other given waypoint. This will be the basis for automatic analysis of a file. We scan the file and this code will allow us to simply output the elapsed times for any legs the recorded trip happens to have taken. We'll probably want to store those in a file or database for automatic analysis, too. | Not done |
| Hash points | Change code to store data in a hash. There should be a hash with the same contents for each of: $this_point, $last_point, $start_point, $last_movement_point, $last_stop_point, and any other points, where each contains the same stuff, like raw data, speed, timestamps, etc. | Completed (20051117-19h00) |
| Auto-results page | Update the code that re-generates all the GMaps to also re-build a webpage which contains the information for the waypoints. There should also be a file from which this code can pull any hand-typed special notes about a route. | Completed (20051117-21h00) |
| Marker subroutine | Write a subroutine for placing markers and linking them to informational windows. | Not done |
| Auto-screencap | Fully automate the collection of small map images. This involves spawning a browser window, writing some code that will somehow know when the page is FINISHED loading, and then auto-capturing a screenshot of that page, before using the already-existing scripts to mangle the screencapture into a good image. | Not done |
| Configuration file | Create a configuration file that has things like: default filenames, extensions, etc; configuration variables such as min-max ranges for resolution hunting, various thresholds, constants, etc. All scripts will eventually use the one file to find the bits they need. | Not done |
| changeme | changeme | Not done |
Possible spin-off projects
| Route-Finder | Combine (A) some code that can look at a route to calculate elapsed times for a set of waypoints, using longest-possible segments, (B) some code that can look at a map and put waypoints at intersections, and (C) a helluva lot of route data -- say, collected by automated equipment in taxis. What do you get? The makings of a server-side system that can suggest fastest routes from A to B using empirical data. |
A new version of the script is complete, with variable resolution. I use a formula that effectively looks at changes in bearing and speed[3] to decide how many data points to include. In the graph to the right, I've plotted the percentage of data points used in the map as a function of the selected delta-speed. As you can see, resolution decreases exponentially, and somewhere between 10 and 20 is a good number to use.
