Feature s3 write csv

[zorge69]

Description

• Added write_csv to save a Pandas df to s3 bucket using boto3 client.

Added a function write_csv to rdsa_utils\cdp\helpers\s3_utils.py. This function utilises Pandas to_csv method and can potentially take any keyword arguments of it. The content of the dataframe is initially serialised into a StringIO buffer. Then, we use the boto3 client's method put_object.

Unit-testing is done using the mocking, via moto library. Three unit tests are added: that True is returned if successful, that the written CSV can be read back and gives the same dataframe, and a negative test, such as if it fails, False is returned. The failure is realised by trying to apply to_csv to a dictionary, which does not have this method.

Type of change

You can delete options that are not relevant.

• New feature - non-breaking change

Checklist:

• I have performed a self-review of my own code.
• I have commented my code appropriately, focusing on explaining my design decisions (explain why, not how).
• I have made corresponding changes to the documentation (comments, docstring, etc.. )
• I have added tests that prove my fix is effective or that my feature works.
• New and existing unit tests pass locally with my changes.
• I have updated the change log.

Peer review

Any new code includes all the following:

• Documentation: docstrings, comments have been added/ updated.
• Style guidelines: New code conforms to the project's contribution guidelines.
• Functionality: The code works as expected, handles expected edge cases and exceptions are handled appropriately.
• Complexity: The code is not overly complex, logic has been split into appropriately sized functions, etc..
• Test coverage: Unit tests cover essential functions for a reasonable range of inputs and conditions. Added and existing tests pass on my machine.

Review comments

Suggestions should be tailored to the code that you are reviewing. Provide context.
Be critical and clear, but not mean. Ask questions and set actions.

These might include:

• bugs that need fixing (does it work as expected? and does it work with other code
  that it is likely to interact with?)
• alternative methods (could it be written more efficiently or with more clarity?)
• documentation improvements (does the documentation reflect how the code actually works?)
• additional tests that should be implemented
  • Do the tests effectively assure that it
    works correctly? Are there additional edge cases/ negative tests to be considered?
• code style improvements (could the code be written more clearly?)

Further reading: code review best practices