DEV Community

Cover image for Transfer Data from CSV files to Deta Bases + Alternative solution.
Fredy Somy
Fredy Somy

Posted on

Transfer Data from CSV files to Deta Bases + Alternative solution.

#database #productivity #python #deta

What are we going to do.

First of all , lets go through this in two phases.

  • Convert csv to json
  • Upload converted json to Deta base

Prerequisites

  • Create a free account in Deta
  • Create a new project (Note down the project key, we need it)

Lets start.

Imagine having a csv file as

name,age fredy,34 god,34345345 mannu,34
Enter fullscreen mode Exit fullscreen mode

We want it like:

[{'name': 'fredy', 'age': '34'},{'name': 'god', 'age': '34345345'} ,{'name': 'maanu', 'age': '34'} ]
Enter fullscreen mode Exit fullscreen mode

Lets start by importing csv module 

import csv
Enter fullscreen mode Exit fullscreen mode

Read the file contents:

arr=[] with open (path) as csvFile: csvReader = csv.DictReader(csvFile) for csvRow in csvReader: arr.append(csvRow)
Enter fullscreen mode Exit fullscreen mode
  • We are making a list named arr for appending all the json extracted from csv. 
csvReader = csv.DictReader(csvFile)
Enter fullscreen mode Exit fullscreen mode

  • This creates a csvreader object which maps each row to a dictionary.

  • We can now use the csvreader to extract the dictionary.

for csvRow in csvReader: arr.append(csvRow)
Enter fullscreen mode Exit fullscreen mode

  • The above loop takes each dictionary from csvreader object and append it to list arr,which i mentioned earlier.

  • We can now print the list arr to see the json data.

>> print(arr) [{'name': 'fredy', 'age': '34'},{'name': 'god', 'age': '34345345'} ,{'name': 'maanu', 'age': '34'} ]
Enter fullscreen mode Exit fullscreen mode

Csv file is converted , lets upload it to Deta Base

  • Install deta module. 
pip install deta
Enter fullscreen mode Exit fullscreen mode
  • Lets import it. 
from deta import Deta
Enter fullscreen mode Exit fullscreen mode
  • Initialize with a Project Key( which we get while creating a project) 
deta = Deta(projectkey) database = deta.Base(db) for each in arr: datafile.insert(each) print("Uploaded {}".format(each))
Enter fullscreen mode Exit fullscreen mode
  • Connect or create a database in Deta 
database = deta.Base(db)
Enter fullscreen mode Exit fullscreen mode 
for each in arr: datafile.insert(each) print("Uploaded {}".format(each))
Enter fullscreen mode Exit fullscreen mode

  • We are iterating through the list arr and datafile.insert(each) inserts each data into the database.

  • Finally we have this in Deta Base UI.

deta

Final Code 

def csvtodeta(id,path,db): deta = Deta(id) datafile = deta.Base(db) print("Reading data from {}".format(path)) arr=[] with open (path) as csvFile: csvReader = csv.DictReader(csvFile) for csvRow in csvReader: arr.append(csvRow) for each in arr: datafile.insert(each) print("Uploaded {}".format(each)) print("succesfully uploaded {} data to {} Base".format(len(arr),db))
Enter fullscreen mode Exit fullscreen mode

Alternative Solution

I created a python CLI tool to do the same.

pip install csvtodeta
Enter fullscreen mode Exit fullscreen mode
  • With just one command the data of csv file is uploaded to Deta🎉. 
$ csvtodeta --id yourprojectkey --path path/to/csv.csv --db detabasename
Enter fullscreen mode Exit fullscreen mode
  • id is your project key from deta
  • Provide the path name of csv file.
  • Provide database name of the Base you want to create or connect.
  • Tada...🎉🎉 The data is uploaded. 
$ csvtodeta --id 45345dhsgh3rjdf2ur34hhwf --path src/deta.csv --db detabasename Reading data from src/deta.csv Uploaded {'name': 'fredy', 'age': '34'} Uploaded {'name': 'god', 'age': '34345345'} Uploaded {'name': 'maanu', 'age': '34'} succesfully uploaded 3 data to detabasename Base
Enter fullscreen mode Exit fullscreen mode

Github Repo

Thank you for reading this blog. If you have any doubts or if there is any wrong explanation please comment down here.

Top comments (0)