Python Trivia Post: enumerate()

Python has a very useful inbuilt function, which is called enumerate(). This is needed whenever you need the numbering of a certain element. Suppose you have a text file or a list or element which contains multiple elements , the enumerate() function can be used to numbering them. As in the example below -I have a file which has multiple lines of text. I can number each line by using this simple technique -Here we open a file and read the lines but while reading we can count the line no by using enumerate.

                                               abc=open('abc.txt')

                                               for count,text in enumerate(abc.readlines()):

                                                   print (count,text)

Also, we can enumerate lists and dictionary with this function in the same way.

Python Trivia Post: .format() function numbering

Look closely at the code snippet given below - 

days = 8
lyrics = "{} days a week is not enough to love you.But I will not be here {1}"
print(lyrics.format(days,"Sunday"))

Can you guess the output of the program? No. It is not what you are thinking. It is an error code - 

ValueError: cannot switch from automatic field numbering to manual field specification

why? As the .format is like an array of elements. So you have to mention the index number to use or you have to leave the numbering blank, so that it can auto-number. You can not use both formats. So in the above program first call of {} and then {1} lead to the error. you can use {0} and {1}  or leave both the places void.

Python Trivia Post: Raw Strings

We generally know that to print a string in python we need to write simply -

print (" You are my sunshine , if you are here , then I am fine !")

But to print a new line we need to write something like this - 

print (" You are my sunshine ,\n if you are here ,\n then I am fine !")

But sometimes we need the raw string, I don't know when but a hacker friend of mine told me it helps while hacking as it is related to raw string searching. To print a raw string write -

print ( r"You are my sunshine ,\n if you are here ,\n then I am fine !") 

and it will print this - 

"You are my sunshine ,\n if you are here ,\n then I am fine !"

Nice one!!!

Difference of Single Aterisk * to Double Asterisk ** in python argument

The single asterisk in argument is given to read multiple arguments at once . In the below code if we call multiply(3,4,5,6) then the code will read all the inputs and then multiply. Otherwise we can not read multiple arguments.

def multiply(*args):
    total=1
    for arg in args:
        total*=arg
    return total

>>> multiply(3,4,5)
....  60
====================== 
If we want to read a dictionary as argument or arbitrary keyword arguments then ** is needed.e.g.

def accept(**kwargs):
    for keyword, value in kwargs.items():
        print("%s -> %r" % (keyword, value))

>>> accept(foo=2,faa=3,gaa="gaa")
('foo', 2)
('faa', 3)
('gaa', 'gaa')
>>> 

====================== 

Reportlab Module

Hey guys, I have found a cool module to print anything to pdf format. The name of the module is reportlab. If you have pip installed in your system then simply write the following in command prompt.

>>> pip install reportlab

The reportlab will automatically be detected anddownloaded if you have a proper internet connection.
Below is a simple code to run and print pdf file form python -

from reportlab.pdfgen import *

def hello(c):

    c.drawString(100,100,"Hello World")

c=canvas.Canvas("hello.pdf")

hello(c)

c.showPage()

c.save()

The code will create a file hello.pdf with "Hello World" written on it. If you want to learn more about reportlab below is the link to learn more - 

https://drive.google.com/file/d/18Ur_Tptn7gqKy10s6cfHOsVuwZEJHX-U/view?usp=sharing

How to Create Your Own Module

The simplest way to create your own module is by using function in Python. Python provide a cest way to create your own module. For example suppose you have function like below -

def myModule(wannasay):
'''This module will say whatever you wanna say'''
return "I wanna say"+str(wannasay)
def calculate(x,y): '''This will add two numbers''' return x+y
    
Save the above code by naming it myModule.py and save it in same directory where you want to create a file which will call it.This is the way we have created the module with methods MyModule and calculate. In the program where we need the module we have to do only one thing is just to import it by the 'import' statement. Lets see how it is done -

import myModule

print "I wanna say something"
myModule.myModule("I am a bad ass lier")

print "Now I wanna calculate"

myModule.calculate(4,6) 


Up, we have imported the file. We put the module file in the same location as the file which called it. We can also put the module in the path of python, where the calling file will automatically search.  Now we used the functions in the module the same way we generally use function in python.

You can find the files here to test yourself -
https://drive.google.com/file/d/0Bx4xTppgmBd-N2wtY2ktVUItQm8/view?usp=sharing
https://drive.google.com/file/d/0Bx4xTppgmBd-SWQ3Uk9GWUZPRUU/view?usp=sharing

How to use python to create Union as in C

The below example shows how to use python to create a union and use it.
from ctypes import *
class barley_amount(Union):
   _fields_ = [
   ("barley_long", c_long),
   ("barley_int", c_int),
   ("barley_char", c_char * 8),
 ]
value = raw_input("Enter the amount of barley to put into the beer vat:")
my_barley = barley_amount(int(value))
print "Barley amount as a long: %ld" % my_barley.barley_long
print "Barley amount as an int: %d" % my_barley.barley_long
print "Barley amount as a char: %s" % my_barley.barley_char  

(ref: gray hat python) 

How to Use C Programming Language with Python

C is one of the first programming language a programmer should learn. But the syntax of the language and semicolons make it hard. But C is the most powerful language ever created. So, we want to use power of C in python and python provides a very good way to do so.

On windows the C runtime is msvcrt.dll which is located in C:\Windows\System32 and on linux it is libc.so.6. We will write a code in Windows and see the result. So Let us C -


from ctypes import *
msvcrt=cdll.msvcrt
message_string="My Name is Anthony\n"
msvcrt.printf("Testing %s",message_string)

In the above code we import all the modules of ctypes, which is responsible for using C in python. Then assign a variable to the dll we need to run the code in the second line. Next, the string we want to print. In the next statement we use pure C function printf() to print our message.

Similarly for Linux OS we need to do a little change. Just replace the second statement to have that libc.so.6. e.g.


from ctypes import *
libc = CDLL("libc.so.6")
message_string="My Name is Anthony\n"
msvcrt.printf("Testing %s",message_string)

That's all for today. In the next entry we'll learn about structure and Union.

(ref: Gray Hat Python)

Extracting Email id Using Python Regular Expression

Regular expression is handy for extracting important part from a document or a string.  Suppose you want to extract email id from a string then it can be easily done by '\S+@\S+' this expression. Where \S represents non-whitespace character, then followed by an @ sign and again by \S;as actually it is in a generic email id. So , let's get the code and extract email id - 

import re

str = 'I have tried to email you at medhrk@gmail.com but it did not work so I send the mail to you by using your rediff account idhrk@rediffmail.com  at  @2PM'

lst = re.findall('\S+@\S+', str)

print lst  

The output will be  - ['medhrk@gmail.com',ídhrk@rediffmail.com],  

Copying with Pyperclip

Pyperclip is a python module for copying and pasting clipboard content. pyperclip.copy() and pyperclip.paste() functions are the main functions of the modules and works in the same manner as the name implied. Peruse the below code and try to grasp what the code will do -

import pyperclip

while True:
    fh=open('allcopies1.txt','a+')    
    text=str(pyperclip.paste())
    if text not in str(fh.read()):
        fh.write(text)
        fh.write("\n\r\n")
    else:
        print "Already Copied"
        fh.close()
        break
fh=open('allcopies1.txt','r')
for word in fh:
    print word
       

This program is useful when you need to have a file for storing all the copied content. Make sure the file has a 777 permission. After opening the file we check if the content of the clipboard copied previously? If not then it adds the clipboard content to the file.
A small but useful tool for big future programs. Can you name me instances when it can be used.

ref: Automate boring stuff by python.

Web Scraping with Python

The main modules which are needed for web related operations are urllib, urlopen, requests, selenium. Other than this four there are other main and supporting modules like beautifulsoup, webbrowser are there. so, let's start with urllib  -

Follow the below code closely -

import urllib
ts=urllib.urlopen("http://pythonbeginner.in")
for textscript in ts:
    print textscript

This code first fetches the whole content of a website and display them line wise. As simple like that. In web scraping, we have to find a given information in a web page. The projects I have got are mostly something like  -  go through the website on a regular basis and update the price of a product accordingly. Or, it can be useful in share market auto trading etc. 

Suppose, we have to find how many times the word 'python' is used in the code. Then what will we do? Web scrapers often face problems like this. Let's look at the second code - 

import urllib,re

ts=urllib.urlopen("http://pythonbeginner.in")

count=0

for line in ts:

    if re.search('python',line):

        print line

        count+=1

print count

Remember the codes from the entry Regular Expression. Here I used the same code to find out the number of times the word 'python' occurs. It found out that I have used it 38 times on the website main page😮.

You can open any page in your default browser with the module webbrowser. Look at the example -

import webbrowser
webbrowser.open("www.yahoo.com")

But for dynamic web pages where the pages are created by ajax, javascript or any other run-time programs then the web scraping becomes difficult. As in the above example, we are getting a static web page, not a dynamic one. To get dynamic values we need to use selenium. I will discuss it in the next blog entry. Wait for it.

Masking Columns in a database

This simple code was one of my project in fiverr.com, a project worth $20. In this code we have to select a database, find certain values in a column according to id and then replace the value by 'x' or numbers by 'y', so that it will be safe. The other part of the code is to keep the original data in a backup file so that we get the data when needed. Look at the code , and see the comments for better understanding of the code

-

See below is the program named databasemask.py , a program which is responsible for masking and backing up the data which will be replaced. Here, in this particular problem we have the database with tables album,customer and invoice , so we declare three dictionaries to hold their values. Lets divulge the secret of the code with comments now -


##As Described above they are for the tables in the database
album = {}
customer = {}
invoice = {}

class Cipherise(): 
"""This class  is for replacing characters with 'x' and numbers with 'y' for a particular column of the table"""
    def __init__(self, key, *the_phrase):
##Initialize with key, and the_phrase, as the_phrase can be a dictionary or a list we use *
        self.num = key
        self.the_phrase = the_phrase
        self.orginal_phrase = the_phrase #retain the initial value in the orginal_phrase value
        self.converted_phrase = ""#initialize converted_phrase with blank string value 

    def __str__(self):
        return """This is for substituting characters with X and numeric with Y"""

    # def convert(self): #You can use this function if you a pass only string ,not a list  to the database
    #     for i in self.the_phrase:
    #
    #         try:
    #             i = int(i)
    #             i = 'Y'
    #             self.converted_phrase += i
    #         except:
    #             i = 'X'
    #             self.converted_phrase += i
    #
    #     return self.converted_phrase

    def convertall(self): ##This is responsible for converting
        maskedlist = [] ## we create an empty masked list
        for phrase in self.the_phrase: ##we peruse through the phrase, remember here the dictionary is supplied
            # print "the phrase as whole",self.the_phrase #To verify what is passed
            for singlephrase in phrase: #Now we peruse through
                self.converted_phrase = '' #start with an empty phrase and then add characters
                for singlechar in singlephrase:
                    try:
                        singlechar = int(singlechar) #See if it is convertible to integer or not
                        singlechar = 'Y' #if yes, then replace it by Y
                        self.converted_phrase += singlechar #Add the value to converted_phrase
                    # print self.converted_phrase
                    except:
                        if singlechar==" ": #Except add X as a replacement of any character
                            singlechar=" "
                            self.converted_phrase += singlechar
                        else:
                            singlechar="X"
                            self.converted_phrase += singlechar

                print (singlephrase)

                maskedlist.append(self.converted_phrase) #Appending the list to maskedlist List variable

        return maskedlist #Returning the masked list from the function

    def getoriginal(self):
        return self.orginal_phrase #Return the original phrase if needed

    def createdictionary(self):#Testing purpose, as here we have a album table this will create an album backup
        global album
        album[self.num] = self.orginal_phrase

    def getdictionary(self):
        global data
        return data


def getbackupalbum():
    f = open('album.txt', 'a+')
    for i in album:
        f.write(str(i) + "," + str(album[i]) + "\n")
    f.close()


def getbackupcustomer():
    f = open('customer.txt', 'a+')
    for i in customer:
        f.write(str(i) + "," + str(customer[i]) + "\n")
    f.close()


def getbackupinvoice():
    f = open('invoice.txt', 'a+')
    for i in invoice:
        f.write(str(i) + "," + str(invoice[i]) + "\n")
    f.close()








from databasemask import *
import sqlite3 as db


scope=str(input("Enter the File Name: "))
database=str(input("Enter the database name :"))
try:
    dbase=db.connect(database)
    cur=dbase.cursor()
except:
    print ("No Such Db")

#Divide the file after reading
def update_table(table,column):
    pkid=str(table)+"Id"
    cur.execute('select %s,%s from %s'%(pkid,column,table))
    data=cur.fetchall()
    for data in data:
        id=data[0]
        #print id
        data=(data[1])
        cipherise=Cipherise(id,(data,))
        maskeddata=cipherise.convertall()
        maskeddata=maskeddata[0]
        id=int(id)
        print (maskeddata)
        cipherise.createdictionary()
        table=str(table)
        try:
            cur.execute('update "%s"'%table +'set "%s"'%column+'="%s"'%maskeddata+' where "%s"'%pkid+'="%d"'%int(id))
        except:
            print ("Error")

        dbase.commit()
    print ("Job Done")

try:
    f=open(scope,'r')
except IOError:
    print ("No such file")
for line in f:
    columnames ={}
    line=line.strip()
    try:
        line=line.split(':')
        table_name=line[0]
        column_names=line[1]

        try:
            column_names=column_names.split(',')
            for column in column_names:
               update_table(table_name,column)
    #             columnames[i]=column_names[i]
        except:
            columnames.append(column_names)
     #       continue
    #     print columnames
    except:
        continue
f.close()
dbase.close()

Cyberoam Firewall - How to block sites and url

The procedure to block unwanted websites/ip address and ports by a cyberoam firewall is as follows -

• First login and then go to objects > hosts . Add a blockIPList as I have done in the above picture. Click ok after adding all the IPs you want to block.

• In the same way go the object > services . Add the ports you want to block. As I have created a blocked ports.

• Then go to Firewall > Rule. Under the LAN to WAN policy create a new rule. As the picture above Enter the services and hosts as previously created.

Now  if you want to simply block a website from your network , do the following steps - 

• Go to web filter > category. Create a category by clicking add category and then adding the websites you want to block.
• Next add the category to your web policy and block connection from those sites.

About Python

Python is a multi function programming language

- this means it can be used to virtually every program. So, in this blog we will try to know Python and how to apply the language to modern day life. But first of all we need to understand the language.

There are many tutorials available in the web. You must visit those tutorials and here is my version of the tutorial. You can learn from here too, if you bear with me.

You can also learn python from sites like -

• docs.python.org
• tutorialspoint
• Codecademy
Or you can use my own site -
• Pythonbeginner

Install Python:

Install Python by going to the site - www.python.org and download the python 2.7  or 3. Here I am concentrating on Python 2.7. So it is advisable to download the later one.
For windows you have to install Python actually, Python is already installed in Linux or Mac. Just extract the downloaded file and double click setup. exe.

How to check python is  installed properly:
For Linux just open the terminal and type Python. Of out shows a reply message with version on it and a '>>' sign, then you are OK.
For windows, open command prompt and type Python.. That's it. If there is an error like command not found, then set the part variable properly or see if you are in proper directory.

After installation of the python from the instruction above you are ready to rock in the python world. You need to download some modules for required project when needed. You can get all the module in github so no worries in this respect.
Why python is Multi-function Programming Language?
Python can be used in many aspects of our every programming needs- from core programming options where we program different hardware to sophisticated web development , we can use python. Python is in its core is made to do tasks without thinking much about syntax of the programming, and concentrate on the semantics of the program. We will start from our very next blog - let's start by saying 'Hello World'.

Type Conversion in Python

How to switch data types in python: Type Conversion

The type conversion method is very much at per with other programming language in python. You just have to write the type you want to convert to. Below are the example  -

>>> a=5  # An integer
>>> a=float(a)
>>> a
5.0
>>> a+5
10.0 #A float from integer and float addition, in fact you will get all the values from float to int as float
>>  a=int(a) #to convert back again to int
>>> a=str(a) #Converting to str
>>> a
'5.0 ' #Converted the float to character or string
>>>  a*5 #concatenate the string five times
 '5.05.05.05.05.0'
>>> a=int(a)
Traceback (most recent call last):
  File "<pyshell#41>", line 1, in <module>
    a=int(a)
ValueError: invalid literal for int() with base 10: '5.0'

>>> a= float(a)
>>>a 
>>> 5.0 #back to float but can convert directly to int 
>>> a=int(a)
5 # Systematic conversion - from str to float to int, so bear in mind that sometimes the conversion pattern depends on the value you are converting.

Now an interesting part, we can always convert the immutable tuple to mutable list in simple way, just like float to integer and vice-versa.
>>> a= (2,3,4,5)
>>> a=list(a)
>>> a
[2,3,4,5]

And in a smae way -
>>> a=tuple(a)
>>> a
(2,3,4,5)

Python Joke:
#Convert me to anything but my soul remain the same..

Nice Way to Raise Exception - Assert

'assert' keyword in function is a nice way by which we can raise an exception. With using assert there is no humdrum of try-except, and we can still have a smooth code. Look at the example below -

import random
num=random.randint(1,50)
for i in range(1,50):
    assert num!=i,'Yes it is the '+str(i)+'th element'
    print i

Here , I have a condition in assert; if the condition is satisfied throughout the loop then I will have the exception. In this code suppose we have a num=12 then we will have the exception only when the value of i and num are same. Otherwise the loop will continue. So we'll get the value of i 1 to 11 and then exception and the program halts there.

You see assert is a small go through to avoid or create exception, but in bigger problems we need try and except to not only catch the exception but also if found redirect it.[ Next Topic]

Comical Assert:
assert - 'Hey! I don'tzz  wannaa do zzhissss, so I gonna stooop here if I encounterrr the problem ' 
you - "ok, boss!!"

How to take a screenshot using Python

Sometimes we need to take screenshot of  our desktop or webpage , though print screen provide the facility but we can also do with python. I am using python screen shot options to take some screen shot of my web page I created with flask( a light weight website development platform which uses python.). First of all you need to have a module named pyscreenshot.

You can install pyscreenshot  from the pip utility , provided you have installed pip previously. If you don't know about pip, see here  -

After you have installed the pip utility, you can easily install pyscreenshot by using the below command -
c:\> pip install pyscreenshot

It will download and install pyscreenshot module. Now let's see how to take a screenshot -

** First of all create a new file and named it anything you want. I named it imagegrab.py. And write the below line to it. 
      import pyscreenshot

      def imagegrab():
           image=pyscreenshot.grab(bbox=(10,10,510,510)).show()

** Save the file. Now you can run the program through idle but it won't show anything.
** To run the program properly - open the command prompt  from the same directory by holding shift and right click or open command prompt and navigate to the directory of the program.

** Type the following to the command prompt -
    C:\programs> python -m imagegrab
It will show the output like  below -

The screen shot is cropped by bbox funtion with bbox(x1,y1,x2,y2) where (x1,y1) is the location of the top left corner or a starting point and (x2,y2) is the end point. You can also grab the screen shot directly to file by using the pyscreenshot.grab_to_file(filename) function. Write help(pyscreenshot) for more on this topic.

But I am disappointed with this pyscreenshot as I was having difficulties with integrating this simple program to flask. Luckilty matplotlib helped to get the screenshot.

Default Argument - Functions

I have been talking about functions a lot more because function is the most essential part of any high-level programming language. Here we will discuss how can we make functions to do different things depending upon different arguments passed. Take a look at the below example  -

def summ(a=2,b=2):
    c=a+b
    print "Value :%s"%str(c)
   
summ(8,2)
summ(3)
summ()
summ(a=5,b=-3)

Here , The first call to the function summ will add the number 8,2 and print result 10, Then in the next one we have passed only one argument , so the function will skip the first default argument and set a to 3, then it get the default value for b, and add them . So the result will be 5. For the next call to the function , the function will use both the default value and give output 4.  You can also pass a value to function by specifying the variable name( here we have assigned b to -3 and passed).
So by having default value assigned in function declaration we can have sophisticated code with no rigidity.

A Better Understanding of Function

After discussing the basics of function in my blog some months back in this link - http://cseatglance.blogspot.in/2015/07/introduction-to-function-python.html , I want to emphasize more about function  in this blog entry. So, first start by peruse through the code below, and guess the output -

#function inside function
#!/usr/bin/python

def total(initial=5,*numbers,**keywords):
    count=initial
    for number in numbers:
        count+=number
    for key in keywords:
        count+=keywords[key]

    def total(count):
        count-=10
        print "Inside me",count
        return count
    total=total(count)
    print "the total",total
    return count

print (total(2,10,12,veg=50,lal=45))


can you guess the output of the program? Here we have one function which has a three initial passed argument and inside it we have another function with same name but have only one argument passed. Among the arguments the initial is set to default value of five and then the tricky arguments passed. the first one with one * will take all the positional arguments
from that point till the end  collected as a list called 'numbers'. The ** will take the input collected as dictionary. See the count+=keywords[key] option. Basically the function takes the input as a list and dictionary values and then add them. 

So, when we pass the values we get only the numeric one and add them.  From the program you can see when write the last print statement it calls the the total with three arguments. Then it adds up the inputs and calls the inner function total, which in turns subtract the value of count with 10. 

In this particular program all the values are added by the outer total. then it subtracted by the inner total function. But we retain the value of outer total function and not of the inner one. So, the output of the program will be -

>>Inside me 109
the total 109
119 

 
 


 

Use your old monitor with raspberry pi

I have an old monitor, a CRT monitor, unused and placed in my junk room. But the problem with that is it is a monitor with only VGA port. So, first of all I bought a HDMI to VGA converter from Amazon. It cost me only 230 rupees.
After getting that converter I directly connect it to the raspberry and a soundbox with the converter's audio output. The sound did not work first but when I made a shall change in the config file it worked fine. You have to just uncomment the lines with HDMI and it will work fine.

Some snaps of my new raspberry PC.