Computes the Z-Score for a given variable in an array of objects. The function adds a new key with the format zScoreVariable in each object.
// Let's say we have a data set like this.
const data = [
{ grade: 1 },
{ grade: 4 },
{ grade: 7 },
{ grade: 2 },
{ grade: 6 },
]
// We compute the Z-Score for the variable grade.
addZScore(data, "grade")
// We now have the key zScoreGrade in the data.
// [
// { grade: 1, zScoreGrade: -1.3155870289605438 },
// { grade: 4, zScoreGrade: 0 },
// { grade: 7, zScoreGrade: 1.3155870289605438 },
// { grade: 2, zScoreGrade: -0.8770580193070292 },
// { grade: 6, zScoreGrade: 0.8770580193070292 },
// ]
+addZscore | Journalism - v1.15.1 Function addZscore
- add
Computes the Z-Score for a given variable in an array of objects. By default, the function adds a new key zScore in each object, but you can change it by passing this option as the last parameter { newKey: "myNewKey" }.
+Parameters
- data: Record<string, unknown>[]
- variable: string
- options: {
newKey?: string;
} = {}Optionalnew
Returns Record<string, unknown>[]
Example: Basic example
// Let's say we have a data set like this.
const data = [
{ grade: 1 },
{ grade: 4 },
{ grade: 7 },
{ grade: 2 },
{ grade: 6 },
]
// We compute the Z-Score for the variable grade.
addZScore(data, "grade")
// We now have the key zScore in the data.
// [
// { grade: 1, zScore: -1.3155870289605438 },
// { grade: 4, zScore: 0 },
// { grade: 7, zScore: 1.3155870289605438 },
// { grade: 2, zScore: -0.8770580193070292 },
// { grade: 6, zScore: 0.8770580193070292 },
// ]
// If you want a specific new key, use the options.
addZScore(data, "grade", { newKey: "zScoreGrade"})
// We now have the key zScoreGrade in the data.
// [
// { grade: 1, zScoreGrade: -1.3155870289605438 },
// { grade: 4, zScoreGrade: 0 },
// { grade: 7, zScoreGrade: 1.3155870289605438 },
// { grade: 2, zScoreGrade: -0.8770580193070292 },
// { grade: 6, zScoreGrade: 0.8770580193070292 },
// ]
-
Compute the adjusted to inflation amount of money, based on the Consumer Price Index. The options (last parameter) are optional.
+Compute the adjusted to inflation amount of money, based on the Consumer Price Index. The options (last parameter) are optional.
// $100 in 1914 (CPI of 6.0) to 2023 value (CPI of 156.4)
const adjustedAmount = adjustToInflation(100, 6.0, 156.4, { decimals: 0 })
// returns 2607 dollars
Optionaldecimals?: numberOptionaldecimals?: numberReturns an array of objects from an object made of arrays.
+Returns an array of objects from an object made of arrays.
For example, this data...
{
keyA: ["a", "b", "c"],
keyB: [1, 2, 3],
}
[
{ keyA: "a", keyB: 1 },
{ keyA: "b", keyB: 2 },
{ keyA: "c", keyB: 3 },
]
Creates folders recursively if they don't exist.
+Creates folders recursively if they don't exist.
// Creates folders if they don't exist
createDirectory("./data/json")
// This will give the same result. A file with an extension at the end of the path will be ignored.
createDirectory("./data/json/items.json")
Convert an array of objects into CSV format.
+Convert an array of objects into CSV format.
const data = [
{ firstName: "Graeme", lastName: "Bruce" },
{ firstName: "Nael", lastName: "Shiab" },
]
const csv = dataAsCsv(data)
// Returns "firstName,lastName\nGraeme,Bruce\nNael,Shiab"
Returns an object made of arrays from an array of objects.
+Returns an object made of arrays from an array of objects.
For example, this data...
[
{ keyA: "a", keyB: 1 },
{ keyA: "b", keyB: 2 },
{ keyA: "c", keyB: 3 },
]
{
keyA: ["a", "b", "c"],
keyB: [1, 2, 3],
}
Compute the distance in kilometres based on longitude and latitude. The options (last parameter) are optional.
+Compute the distance in kilometres based on longitude and latitude. The options (last parameter) are optional.
const distance = distance(-73.66, 45.51, -79.43, 43.66, { decimals: 0 })
// returns 501
Optionaldecimals?: numberOptionaldecimals?: numberFormat a Date as a string with a specific format and a specific style. To format as UTC Date, set the utc option to true.
+Format a Date as a string with a specific format and a specific style. To format as UTC Date, set the utc option to true.
const date = new Date("2023-01-01T01:35:00.000Z")
const string = formatDate(date, "Month DD, YYYY, at HH:MM period", { utc: true, abbreviations: true })
// returns "Jan. 1, 2023, at 1:35 p.m."
Options can be passed as the last parameter. Pass {style: "rc"} to parse dates in French.
-Optionalabbreviations?: booleanOptionalnoOptionalstyle?: "cbc" | "rc"OptionaltimeOptionalutc?: booleanOptionalabbreviations?: booleanOptionalnoOptionalstyle?: "cbc" | "rc"OptionaltimeOptionalutc?: booleanFormat a number with a specific style.
+Format a number with a specific style.
const string = formatNumber(1234.567, { sign: true, round: true })
// returns "+1,235"
Optionaldecimals?: numberOptionalfixed?: booleanOptionalnearestOptionalprefix?: stringOptionalround?: booleanOptionalsign?: booleanOptionalsignificantOptionalstyle?: "cbc" | "rc"Optionalsuffix?: stringOptionaldecimals?: numberOptionalfixed?: booleanOptionalnearestOptionalprefix?: stringOptionalround?: booleanOptionalsign?: booleanOptionalsignificantOptionalstyle?: "cbc" | "rc"Optionalsuffix?: stringConvert longitude and latitude to x,y,z coordinates based on a given radius. The options (last parameter) are optional.
+Convert longitude and latitude to x,y,z coordinates based on a given radius. The options (last parameter) are optional.
const coords = geoTo3D(-73.5674, 45.5019, 1, { decimals: 2})
// returns { x: -0.67, y: 0.71, z: 0.2 }
You can pass { toArray: true } to return an array instead of an object.
-Optionaldecimals?: numberOptionaltoOptionaldecimals?: numberOptionaltoReturn the closest item of a list based on longitude and latitude. The options (last parameter) are optional. If addDistance is true and geoItems have a properties key, the distance will be added to the properties.
+Return the closest item of a list based on longitude and latitude. The options (last parameter) are optional. If addDistance is true and geoItems have a properties key, the distance will be added to the properties.
const geoItems = [
{name: "Montreal", lon: -73.66, lat: 45.51 },
{name: "Toronto", lon: -79.43, lat: 43.66 },
]
const ottawa = {lat: 45.37, lon: -75.71}
const closest = getClosest(
ottawa.lon,
ottawa.lat,
geoItems,
d => d.lon,
d => d.lat,
{ addDistance: true, decimals: 3 }
)
// return { name: "Montreal", lon: -73.66, lat: 45.51, distance: 160.694 }
OptionaladdOptionaldecimals?: numberOptionaldistance?: numberOptionalproperties?: { Optionaldistance?: numberOptionaladdOptionaldecimals?: numberOptionaldistance?: numberOptionalproperties?: { Optionaldistance?: numberExtracts detailed informations from a geoTIFF that can be used with the getGeoTiffValues function. Just for NodeJS and similar runtimes.
+Extracts detailed informations from a geoTIFF that can be used with the getGeoTiffValues function. Just for NodeJS and similar runtimes.
const geoTiffDetails = await getGeoTiffDetails("./some-file.tif")
const value = await getGeoTiffValues(45.50, -73.57, geoTiffDetails)
Extracts values at specific lat/lon coordinates from a geotiff. Works with the values returned by the getGeoTiffDetails function.
+Extracts values at specific lat/lon coordinates from a geotiff. Works with the values returned by the getGeoTiffDetails function.
const geoTiffDetails = await getGeoTiffDetails("./some-file.tif")
const value = await getGeoTiffValues(45.50, -73.57, geoTiffDetails)
Returns the data from an HTML table as an array of objects. The first parameter is an url. The second parameter is an optional object specifying a css selector and/or an index.
+Returns the data from an HTML table as an array of objects. The first parameter is an url. The second parameter is an optional object specifying a css selector and/or an index.
// This would parse the data from the fourth
// table with the class name data-table.
const data = await getHtmlTable("your-url-here", {
selector: ".data-table",
index: 3
})
Optionalindex?: numberOptionalselector?: stringOptionalindex?: numberOptionalselector?: stringCalculate Humidex Factor in Celsius given the temperature in Celsius and humidity percentage. +
Calculate Humidex Factor in Celsius given the temperature in Celsius and humidity percentage. In case the calculated humidex is less than the given temperature, it returns temperature itself.
const humidex = getHumidex(30, 70); // returns 41
This is using the formula from the Canadian Centre for Climate Services.
-Determines the current season based on a date (current date by default). Options include hemisphere (northern by default) and type (astronomical by default).
-Optionaldate?: DateOptionalhemisphere?: "northern" | "southern"Optionaltype?: "meteorological" | "astronomical"Determines the current season based on a date (current date by default). Options include hemisphere (northern by default) and type (astronomical by default).
+Optionaldate?: DateOptionalhemisphere?: "northern" | "southern"Optionaltype?: "meteorological" | "astronomical"Returns the data of a Google Sheet.
+Returns the data of a Google Sheet.
By default, this function looks for the API key in process.env.GOOGLE_PRIVATE_KEY and the service account email in process.env.GOOGLE_SERVICE_ACCOUNT_EMAIL. If you don't have credentials, check this.
// Fake url used as an example.
const sheetUrl = "https://docs.google.com/spreadsheets/d/nrqo3oP4KMWYbELScQa8W1nHZPfIrA7LIz9UmcRE4GyJN/edit#gid=0";
// Returning the data as an array of objects.
const data = await getSheetData(data, sheetUrl);
// Same but skipping first row.
const data = await getSheetData(data, sheetUrl, { skip: 1});
// You have an option to return the data as a CSV string. Useful if you just want to write the data somewhere.
const csv = await getSheetData(data, sheetUrl, { csv: true });
// If your API email and key are stored under different names in process.env, use the options.
const csv = await getSheetData(data, sheetUrl, { apiEmail: "GG_EMAIL", apiKey: "GG_KEY" });
OptionalapiIf your API key is stored under different names in process.env, use this option.
Optionalcsv?: booleanIf true, the function will return a CSV string instead of an array of objects.
Optionalskip?: numberThe number of rows to skip before parsing the data. Defaults to 0.
-Returns the data from a Statistics Canada table as an array of objects. The first parameter is the pid value that can be found in the table url. The second parameter is an optional object specifying:
+Returns the data from a Statistics Canada table as an array of objects. The first parameter is the pid value that can be found in the table url. The second parameter is an optional object specifying:
const data = await getStatCanTable('98100001')
Optionaldebug?: booleanOptionallang?: "en" | "fr"OptionalreturnOptionalskipOptionaldebug?: booleanOptionallang?: "en" | "fr"OptionalreturnOptionalskipCalculates the mortgage insurance premium based on the property value and down payment. The returned value is rounded to the nearest integer. Based on the Financial Consumer Agency of Canada calculator.
+Calculates the mortgage insurance premium based on the property value and down payment. The returned value is rounded to the nearest integer. Based on the Financial Consumer Agency of Canada calculator.
// Returns 19_000
const insurancePremium = mortgageInsurancePremium(500_000, 25_000)
The price of the purchased property.
The amount of money paid upfront.
-Calculates the maximum purchase price (and other variables) for a property a person can afford and the related mortgage it would qualify for based on annual income, down payment, mortgage interest rate, and additional options.
+Calculates the maximum purchase price (and other variables) for a property a person can afford and the related mortgage it would qualify for based on annual income, down payment, mortgage interest rate, and additional options.
// With an annual income of $100,000, a down payment of $25,000, and a rate of 5.25%.
const results = maxMortgageAmount(100_000, 25_000, 5.25)
// results = {
// annualIncome: 100000,
// downPayment: 25000,
// rate: 5.25,
// rateTested: 7.25,
// purchasePrice: 307000,
// mortgageAmount: 293280,
// insurancePremium: 11280,
// monthlyMortgagePayment: 2099.65,
// grossDebtServiceRatio: 0.32,
// totalDebtServiceRatio: 0.32,
// reason: "debt limit",
// monthlyDebtPayment: 0,
// monthlyHeating: 175,
// isHeatingEstimate: true,
// monthlyTax: 385,
// isTaxEstimate: true,
// monthlyCondoFees: 0,
// }
OptionalmonthlyThe monthly debt payment of the borrower. Defaults to 0.
OptionalmonthlyThe monthly heating cost. Defaults to $175.
OptionalmonthlyThe monthly property tax. Default to 1.5% of the purchase price.
-Returns fixed rate mortgage payments in an array. Each payment is an object with the paymentId, the payment amount, the interest and capital portions of the payment, the remaining mortgage balance, and the total amount paid, total interest paid, and total capital reimbursed so far. The calculations have been tested for Canada, which requires fixed rate mortgages to be compounded semi-annually by law. But you can change the annualCompounding in the options.
+Returns fixed rate mortgage payments in an array. Each payment is an object with the paymentId, the payment amount, the interest and capital portions of the payment, the remaining mortgage balance, and the total amount paid, total interest paid, and total capital reimbursed so far. The calculations have been tested for Canada, which requires fixed rate mortgages to be compounded semi-annually by law. But you can change the annualCompounding in the options.
If the amortizationPeriod is smaller than the term, an error is thrown.
These options can be passed in an object as the last parameter:
Calculations are based on https://www.yorku.ca/amarshal/mortgage.htm and https://www.mikesukmanowsky.com/blog/a-guide-to-canadian-mortgage-calculations
-OptionalannualOptionaldebug?: booleanOptionaldecimals?: numberOptionalid?: stringOptionalannualOptionaldebug?: booleanOptionaldecimals?: numberOptionalid?: stringClears a Google Sheet and populates it with new data.
+Clears a Google Sheet and populates it with new data.
By default, this function looks for the API key in process.env.GOOGLE_PRIVATE_KEY and the service account email in process.env.GOOGLE_SERVICE_ACCOUNT_EMAIL. If you don't have credentials, check this.
// The data needs to be an array of objects. The keys of the first object will be used to create the header row.
const data = [
{ first: "Nael", last: "Shiab" },
{ first: "Andrew", last: "Ryan" },
];
// Fake url used as an example.
const sheetUrl = "https://docs.google.com/spreadsheets/d/nrqo3oP4KMWYbELScQa8W1nHZPfIrA7LIz9UmcRE4GyJN/edit#gid=0";
// Clearing the sheet and then populating it.
await overwriteSheetData(data, sheetUrl);
// Same thing but with raw values. Google Sheet won't try to guess the data types and won't format or parse the values.
await overwriteSheetData(data, sheetUrl, { raw: true });
// Adding the UTC date of the update before the data.
await overwriteSheetData(data, sheetUrl, { lastUpdate: true });
// You can also format the date to a specific time zone.
await overwriteSheetData(data, sheetUrl, { lastUpdate: true, timeZone: "Canada/Eastern" });
// The prepend option allows you to add extra text on the first row.
await overwriteSheetData(data, sheetUrl, { prepend: "Contact xxxx.xxxx@gmail.com for more information", lastUpdate: true, timeZone: "Canada/Eastern" });
// If your API email and key are stored under different names in process.env, use the options.
await overwriteSheetData(data, sheetUrl, { apiEmail: "GG_EMAIL", apiKey: "GG_KEY" });
Optionalprepend?: stringText to be added before the data.
Optionalraw?: booleanIf true, Google Sheet won't try to guess the data type and won't format or parse the values.
OptionaltimeIf lastUpdate is true, you can use this option to format the date to a specific time zone.
-Returns the duration as a string in terms of milliseconds, seconds, minutes, hours, and days.
+Returns the duration as a string in terms of milliseconds, seconds, minutes, hours, and days.
// A starting Date somewhere in your code
const startDate = new Date() // or Date.now()
// When you want to know the elapsed duration, pass the start date
const duration = prettyDuration(startDate)
// Returns something like "22 days, 6 h, 3 min, 15 sec, 3 ms"
// If you want to console.log it, set the option log to true
prettyDuration(startDate, {log: true})
// You can also use a prefix and/or suffix
prettyDuration(startDate, {log: true, prefix: "Total duration: ", suffix: " (Main function)"})
// Returns and logs something like "Total duration: 3 min, 15 sec, 3 ms (Main function)"
// If you want to format the duration between two specific dates, use the end option.
prettyDuration(new Date("2024-01-01T17:00:00"), { end: new Date("2024-01-23T23:03:15") })
// Returns "22 days, 6 h, 3 min, 15 sec, 0 ms"
Optionalend?: number | DateOptionallog?: booleanOptionalprefix?: stringOptionalsuffix?: stringOptionalend?: number | DateOptionallog?: booleanOptionalprefix?: stringOptionalsuffix?: stringPublishes the specified Datawrapper chart, table, or map. By default, this function looks for the API key in process.env.DATAWRAPPER_KEY.
+Publishes the specified Datawrapper chart, table, or map. By default, this function looks for the API key in process.env.DATAWRAPPER_KEY.
const chartID = "myChartId"
await publishChartDW(chartID)
// If your API key is stored under a different name in process.env, use the options.
await publishChartDW(chartID, { apiKey: "DW_KEY" })
OptionalapiOptionalreturnOptionalapiOptionalreturnRound a number. By default, round to the nearest integer.
+Round a number. By default, round to the nearest integer.
const string = round(1234.567, { decimals: 1 })
// returns 1,235.6
Optionaldecimals?: numberOptionalnearestOptionalsignificantOptionaltry?: booleanOptionaldecimals?: numberOptionalnearestOptionalsignificantOptionaltry?: booleanSaves an Observable Plot chart as an image. You must use the Plot.dot syntax and install puppeteer (npm i puppeteer).
+Saves an Observable Plot chart as an image. You must use the Plot.dot syntax and install puppeteer (npm i puppeteer).
import * as Plot from "@observablehq/plot"
// The data must be an array of objects.
const data = [{ salary: 75000, hireDate: new Date("2023-12-22") }, ...]
// The Plot options must be wrapped into a function and use the Plot. syntax.
const chart = () => Plot.plot({
marks: [
Plot.dot(data, {x: "hireDate", y: "salary"})
]
})
// Change the extension to .jpeg to get a JPEG file.
const path = "./my-chart.png"
await savePlotChart(data, chart, path)
Returns the OpenGIS Styled Layer Descriptor encoded for an URL. The required parameters are the layer and the color scale.
+Returns the OpenGIS Styled Layer Descriptor encoded for an URL. The required parameters are the layer and the color scale.
// Returns the SLD for the GDPS.ETA_TT layer with a color scale going from blue to red.
const sdl = styledLayerDescriptor("GDPS.ETA_TT", [
{ color: "#550c24", value: 100 },
{ color: "#7f2e34", value: 30 },
{ color: "#c26847", value: 20 },
{ color: "#bdbb7a", value: 10 },
{ color: "#e0e9f0", value: 0 },
{ color: "#97b4cd", value: -10 },
{ color: "#5881a1", value: -20 },
{ color: "#334f60", value: -30 },
{ color: "#21353f", value: -100 },
])
// The sdl can now be used in a WMS request as SLD_BODY
const url = `https://geo.weather.gc.ca/geomet?SERVICE=WMS&VERSION=1.3.0&REQUEST=GetMap&BBOX=-90,-180,90,180&CRS=EPSG:4326&WIDTH=2400&HEIGHT=1200&LAYERS=GDPS.ETA_TT&FORMAT=image/jpeg&SLD_BODY=${sld}`
Unzips a file and outputs the result in a folder.
+Unzips a file and outputs the result in a folder.
unzip("files.zip", "./output/")
unzip("files.zip", "./output/", { deleteZippedFile: true })
OptionaldeleteOptionaldeleteUpdates annotations in on a chart. By default, this function looks for the API key in process.env.DATAWRAPPER_KEY.
+Updates annotations in on a chart. By default, this function looks for the API key in process.env.DATAWRAPPER_KEY.
import { updateAnnotationsDW } from "journalism"
const chartID = "myChartId"
const myAnnotations = [
{
"x": "2024/08/30 01:52",
"y": "14496235",
"text": "This is an annotation!",
},
{
"x": "2024/06/29",
"y": "15035128",
"dy": 50,
"text": "This is also some text, but with an arrow!",
"connectorLine": {
"enabled": true,
"type": "straight",
"arrowHead": "lines"
},
"mobileFallback": false
}
]
await updateAnnotationsDW(chartID, myAnnotations)
// If your API key is stored under a different name in process.env, use the options.
await updateAnnotationsDW(chartID, myAnnotations, { apiKey: "DW_KEY" })
OptionalapiOptionalreturnOptionalapiOptionalreturnUpdates the data of a specified Datawrapper chart, table or map. By default, this function looks for the API key in process.env.DATAWRAPPER_KEY.
+Updates the data of a specified Datawrapper chart, table or map. By default, this function looks for the API key in process.env.DATAWRAPPER_KEY.
Example for a chart.
import { updateDataDW, dataAsCsv } from "journalism"
const chartID = "myChartId"
const data = [
{ salary: 75000, hireDate: new Date("2022-12-15") },
...
]
const dataForChart = dataAsCsv(data)
await updateDataDW(chartID, dataForChart)
// If your API key is stored under a different name in process.env, use the options.
await updateDataDW(chartID, dataForChart, { apiKey: "DW_KEY" })
import { updateDataDW } from "journalism"
const mapID = "myMapId"
const geojson = {
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"properties": {},
"geometry": {
"coordinates": [
[
[
11.127454320325711,
20.34856592751224
],
[
11.127454320325711,
-13.781306861158996
],
[
55.68071875381875,
-13.781306861158996
],
[
55.68071875381875,
20.34856592751224
],
[
11.127454320325711,
20.34856592751224
]
]
],
"type": "Polygon"
}
}
]
}
const dataForMap = {
"markers": [
{
"id": "m1",
"type": "area",
"visible": true,
"exactShape": true,
"fill": true,
"stroke": true,
"properties": {
"fill": "#15607a",
"fill-opacity": 0.2,
"stroke": "#15607a",
"stroke-width": 1,
"stroke-opacity": 1,
"stroke-dasharray": "100000",
"pattern": "solid",
"pattern-line-width": 2,
"pattern-line-gap": 2
},
"feature": geojson
}
]
}
await updateDataDW(mapID, JSON.stringify(dataForMap))
// If your API key is stored under a different name in process.env, use the options.
await updateDataDW(mapID, JSON.stringify(dataForMap), { apiKey: "DW_KEY" })
OptionalapiOptionalreturnOptionalapiOptionalreturnUpdates notes field for a specified Datawrapper chart, table or map. By default, this function looks for the API key in process.env.DATAWRAPPER_KEY.
+Updates notes field for a specified Datawrapper chart, table or map. By default, this function looks for the API key in process.env.DATAWRAPPER_KEY.
import { updateNotesDW, formatDate } from "journalism"
const chartID = "myChartId"
const dateString = formatDate(new Date(), "Month DD, YYYY, at HH:MM period", { abbreviations: true })
const note = `This chart was last updated on ${dateString}`
await updateNotesDW(chartID, note)
// If your API key is stored under a different name in process.env, use the options.
await updateNotesDW(chartID, note, { apiKey: "DW_KEY" })
OptionalapiOptionalreturnOptionalapiOptionalreturnZips multiple files together. To zip an entire folder, pass the folder path as the first parameter. To zip specific files, pass their path as an array of strings. The function will create the path of the zipped file if it doesn't exist.
+Zips multiple files together. To zip an entire folder, pass the folder path as the first parameter. To zip specific files, pass their path as an array of strings. The function will create the path of the zipped file if it doesn't exist.
// Entire folder
zip("./data", "./data.zip")
// Specific files
zip(["./file1.json", "./file2.txt", "./file3.jpg"], "./files.zip")
Computes the Mahalanobis distance from an origin object for each object in an array. The keys in the origin object are the dimensions considered (tested up to four dimensions but should work for more in theory). The function adds the key
+mahaDistin each object in the original array.- Preparing search index...
- The search index is not available
Journalism - v1.15.1Function addMahalanobisDistance
Computes the Mahalanobis distance from an origin object for each object in an array. The keys in the origin object are the dimensions considered (tested up to four dimensions but should work for more in theory). The function adds the key
mahaDistin each object in the original array.If you pass the option
{ similarity: true }, it will add another keysimilaritywhich goes from 1 to 0. The closer similarity is to 1, the closer the item is to the origin.Parameters
similarity?: boolean;
} = {}
Optionalsimilarity?: booleanReturns Record<string, unknown>[]
Example: Basic usage
-Settings