aws: optional "minimal" ingestion mode

[patrislav]

Abstract

One of the ways we can improve AWS ingestion time (#13) is to support a "minimal" mode. In this mode, only a subset of all the data would be ingested, correlating to the resources we support. This will let library users save time and space by ingesting only around 10% (in the case of EC2) of the pricing file.

Implementation details

This can be done by having a new AWS ingestion option, e.g. WithIngestionFilter, that would accept a filtering function. The function would be called for each pricing record, marking the items that should (and shouldn't) be ingested. Terracost might then define two such functions out-of-the-box: a DefaultFilter that ingests everything and a MinimalFilter that only ingests records necessary for the library to work.

The "minimal" mode shouldn't be the default, as it will require re-ingestion each time new resources or attributes of already existing resources become supported.