Graph not showing what was expected - x-axis order changed.

[GilesGibson]

live site
logged on as gilesgibson

One for @dataunity

Vis created from a csv file that was extracted from some Met Police data. It is the Police headcount Lambeth 2012-2014 dataset. The series in the csv file is month by month from Jan 2012 through to June 2014. However, when graphed, the sequence changes to one with all the Jans together, then all the febs, Mars and so on, or roughly like that. Not the order in the csv file.

It may be that the csv file was tacky. In a text editor it looks OK, just wondering what are the rules that the DU followed to change the order from that in the csv file?

[dataunity]

This is down to the data types in the data. If you open the CSV file you'll see it that the x-axis values are strings like "Dec 12". Strings are naturally ordered alphabetically which is why the x-axis looks like the above.

This one requires data cleaning functionality. CSV data doesn't have a 'date' type that you can use - it uses strings or numbers so it needs some extra functionality to set the type when the data is processed. I haven't had a chance to set this up yet with all the backend work so I'll leave open for now.

[GilesGibson]

I suppose I thought that it would not try and re-order text, just leave it as it was, the same order as in the csv file. Does it treat all text as strings? If so how can you get an x-axis that has different text labels to stay in the order you wish it to be? Sorry to be a bit slow in this, just uncertain how your algorithm works.

[dataunity]

It's less the way that I've programmed things, more to do with the way visualisation works with D3. If you look at the y axis, you'll see it starts from zero and goes to higher numbers as you go higher up the screen. It's similar with the x axis, the lowest values appear on the left and higher values go to the right (in other words bar charts are plotted on a cartesian plane).

Text is naturally ordered alphabetically, so values starting with 'A' (like 'April') will be further to the left and values starting with 'Z' will be towards the right, with other letters in between. Hence 'Sept 12' comes after 'Oct 12' when ordered alphabetically, as you're seeing on the chart above.

The process of visualisation normally reorders entries in the data as it's displayed on the screen - it's something desirable. For example you want the y value 50 to appear higher than the value 40 even though the value 50 might have appeared in the data before the value 40.

The reason you see other charts ordered chronologically is that the dates have been converted into numbers (in javascript) before plotting, then a special formatting process has been applied to the labels on the x axis to turn the numbers back into some human readable text (like 'Jan 12'). It's this process we don't have at the moment.

I'm oversimplifying a bit, but basically that's why the values are ordered the way they are.

A word on dates: There's no 'date' type in CSV, which is why the values are stored as a string (different people might store them in different ways). There's problems due to the huge variation in the ways to describe dates even in English - they can be represented in many ways (20/8/2014, 20 August 14, 20 Aug 2014, 2014-08-20).

[djwesto]

Assigning to @GilesGibson because it looks like we're still waiting for a response to @dataunity's point from you sir...

[GilesGibson]

It looks like we will not be able to get a chronological bar graph until new features are added via DU.

[pmackay]

Removing "bug" label as this seems debatable.