A Program to Read Your xls File and Then Create a Database Through Python

I had a problem last week, I needed to convert an xls file and a create a database file form it , so that I can use it in flask architecture and create a web based application to store data. The old data of xls was important, so I can not start afresh without including the data. Below is the way I did the program to solve the problem -

1. First open the xls sheet and then I found and replace all the commas(,) with space , so that when I try to split the lines by comma the commas which are in each cell will be exempted. The next thing is to save the file as .csv(comma separated value) format.
2. Now all we need to do is to write the program which will spit the files couple of times and insert it in the database.

import sqlite3 as db
dbase=db.connect('hostname3')
cur=dbase.cursor()
##cur.execute('drop table if exists hostname')#
##cur.execute("CREATE TABLE hostname(serial_no text,workgroup TEXT,computer_name TEXT,dept TEXT, designation TEXT,location TEXT,ipaddress TEXT,io_box TEXT,system_model TEXT,ms_office TEXT,winxp_cdkey TEXT,os TEXT, ram TEXT, processor TEXT, monitor TEXT,keyboard TEXT, mouse TEXT,printer TEXT,cartridge TEXT,scanner TEXT,ups TEXT,spike_bus TEXT,network_printer_address TEXT);")

fh=open('new.csv','r+')

for line in fh:
    line=line.split('\n')
    for data in line :           
          data=(data[0],data[1],data[2],data[3],data[4],data[5]\
         ,data[6],data[7],data[8],data[9],data[10],data[11],\             
         data[12],data[13],data[14],data[15],data[16],data[17],data[18]\
         ,data[19],data[20],data[21],data[22])
        print (data)
      
        cur.executemany("insert into hostname values (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)",(data,))

fh.close()
dbase.commit()
dbase.close()

I have commented the lines which will be needed if we run the program for the first time. The above program first creates a database and then create a table according to the original excel sheet. Then we open the newly created  csv file , then for iterate through the file and split them when we find a newline. We again iterate through each  line and split them when we find a comma. Then we execute the execute many command which will insert the whole data to the table. Don't run the program with python 2.7 ,instead use python 3 to run the program, otherwise it will give a some UTF encoding error. Python 2.7 is okay with single data insertion but when it comes to the executemany command it showed the error. If you find any solution to run the program in python 2.7 then please comment below. You can also visit my site http://www.pythonbeginner.in for more information.
It saved my time a lot. Hope it will help you too.

Working with CSV file in Python

Python has CSV(Comma Separated Values) module which can be imported when working with csv file. Here in the below program I first created a txt file and then copy the content of the file to the csv file for your understanding. You can get user input and put them directly to CSV file too.
First of all learn some basics about the csv module. Open your IDLE and import csv module and the write dir(csv) to see all the processes embedded with this module -
>>> import csv
>>> dir(csv)
['Dialect', 'DictReader', 'DictWriter', 'Error', 'QUOTE_ALL', 'QUOTE_MINIMAL', 'QUOTE_NONE', 'QUOTE_NONNUMERIC', 'Sniffer', 'StringIO', '_Dialect', '__all__', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__version__', 'excel', 'excel_tab', 'field_size_limit', 'get_dialect', 'list_dialects', 're', 'reader', 'reduce', 'register_dialect', 'unregister_dialect', 'writer']

Please visit the site https://docs.python.org/2/library/csv.html  for detailed discussions of every processes described here. Though some of them is self explanatory and you can always use help(csv.processname) for more informations e.g. >>> help(csv.Dialect)
As this is just a basic program for inputting data to a csv file , I will restrict myself to some basic procedures.
Lets move on to the program;in the program I opened a text file and got some user inputs to the file. I now then open the file again in read mode and extract data from the file. Then I feed the data to a csv file I created in the next part of the program. The program needs many modifications which I have not done intently as I want the readers to modify the program.

#create a csv file with the help of a txt file  -
import csv #importing the csv module
f=open("atextfile.txt",'w')  #opening a file in read mode
entry="" #an empty string
while entry.upper()!='QUIT': #getting user input unless he types 'quit'
    entry=str(raw_input('Write something to file in a Name, age format')) #have the input in entry
    if entry=='quit' or QUIT:
        break #break from loop if quit pressed
    else:
        f.write(str(entry)+'\n') #writing the entry to the file with a new line after each entry
f.close() #closing the file for saving the values  

f=open('atextfile.txt','r') #opening the file again in read mode
data=f.readlines() #readine the data line by line
print data

with open ('anewfile.csv','w') as csvfile: #created a csv file and say it csvfile in the program
    writer=csv.writer(csvfile,delimiter='\n', quoting=csv.QUOTE_ALL) #this is the default csv writer. we have the csv file name, the delimiter which is in our case is newline character and quoting to say to quote all the inputs, don't care if the value is numeric or not.
    for line in data: #Getting to each line of the data
        line=line.split() #splitting the data by white space
       # print line
        writer.writerow(line) #write line to csv file as row basis
    csvfile.close() #closing the file
    print "Successfully Done"
f=open('anewfile.csv','r') #Opening the csv file
rows=csv.reader(f) #read the content to rows
for line in rows : #display each row
    print line

There are many flaws in the program, the main one is that it will not write name and address in a same row, rather it will create a new row for each of the element. Rectifying the problem needs some basic formatting and I will cover that up in my upcoming blog entry.

This program is a basic csv writer , the formatting of rows is not done here. I will do it in the upcoming programs. But it will give you a basic idea about the csv file operations. Peruse through the link I have given above and also go to idle help option to know more about or you can always wait for my next blog entry.