Skip to content

Conversation

@Arpan-Kreeti
Copy link
Collaborator

@Arpan-Kreeti Arpan-Kreeti commented Nov 13, 2021

Description

This PR contains the following changes

Code improvements

Upgrade various dependencies and some code cleanup

Remove timestamp and add datetime column type support

MySQL attempts to convert timezone to UTC when using timestamp columns. In order to avoid this we are now using datetime to store date time data instead of timestamp.

Add support for a custom date or DateTime formats

Fixes #11

By default csv2sql will identify date or datetime of the following patterns YYYY-MM-DD and YYYY-MM-DD hh:mm:ss respectively.

If a csv file contains date or datetime in some other format then they will be imported as varchar by default.

In this PR we add support for specifying custom patterns to import date or datetime data of arbitrary formats.

In order to specify custom patterns for date or datetime use the --custom-date-patterns or --custom-datetime-patterns arguments followed by a string having one or more patterns separated by ;

When using the Web UI for csv2sql enter these pattern strings in the config page under "Custom date patterns" or "Custom datetime patterns".

The patterns should be compatible with Timex directives specified here.

Examples

To parse datetime like 11/14/2021 3:43:28 PM a pattern like {0M}/{0D}/{YYYY} {h12}:{m}:{s} {AM} can be specified

The complete command might look like...

./csv2sql --source-csv-directory "/home/user/Desktop/csvs" --db-connection-string "mysql:root:pass@localhost/test_csv" --custom-datetime-patterns "{0M}/{0D}/{YYYY} {h12}:{m}:{s} {AM}"

From Web UI
image

@Arp-G Arp-G marked this pull request as ready for review November 14, 2021 13:53
@Arpan-Kreeti Arpan-Kreeti merged commit e716a8b into master Nov 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

4 participants