Python at Coffee Table

Basics
Command line arguments stored in sys.argv and get by index sys.argv[0]
comment start with pound(#) sign.
indentation is use for grouping statments like we are using {} in C++ or perl for grouping or making for loops and if conditional statements
compileall tool is use for generate .pyc “byte-compiled” version of the module.
first line of script #! /usr/bin/env python
define source code encoding
# -*- coding: encoding -*-
# -*- coding: iso-8859-15 -*-
setting python startup script by setting an environment variable named PYTHONSTARTUP. this script will run evry time python interpreter start.

command line arguments
len(sys.argv) # number of arguments
sys.argv[0] # script name

looping and condition statements
checking condition use keywords : in, not in, is, is not, and, or 
if statment 
if x>100:
.....
elif x>50:
......
else
....
for statment
for x in alist:
print x
for x in range(len(alist))
print x, alist[x] # this put space between items and make formating

break, continue, else statments in loop

function defination
funcation paramter can have defualt values, can have keyword based call
def func1(param1,param2):
...
Complex Number 
complex(real,img) function
using suffix j or J.  cn=complex(2,5)  or cn = 2+5J
extract complex number cn.real or cn.img

conversion function 
float(),   int(),   long(),   round(12.098,2), abs(11.78)

string 
can have in double quotes("string"), single quotes('string'), matching triple quotes("""" or '''')
create raw string using r like : hello =r"this is raw string \n\"   # it will not treat \n as new line character,
string concate using + operatoer and repeat * operator
hello = "Hello" + "viru"
wrong "This is "+ "wrong"*5
string can accessing using index[2] or slice index[1:10] or negative index [-4], negative index can start counting from right
string methods   len(),str(),format
Unicode string  use prefix u : hello = u"Helllo¦|¿"
change unicode to other encoding using method encode()

List
list is comma seprated values between square bracket[]  : a=["viru",1, 4.5,"this"] # it can contain different types of data.
list can support same kind of index for accessing like strings.
list methods list.append(item), list.extend(L), list.insert(index,item), list.remove(item), list.pop(index), list.sort(), list.count(), list.reverse()
del list[index] : del statement is use for delete entry from list using index or slice index
vec = [2, 4, 6] # list example

sequence
filter(function, sequence) returns a sequence consisting of those items from the sequence for which function(item) is true
map(function, sequence) calls function(item) for each of the sequence’s items and returns a list of the return values.
reduce(function, sequence) returns a single value constructed by calling the binary function function on the first two items of the sequence, then on the result and the next item, and so on.
use sorted() function to sort sequance :    sorted(seq)
range() function generate sequance of number(list).

Tuples
A tuple consists of a number of values separated by commas  : t = 1,3,67,"hello"    empty_tuple=()
Tuples, like strings, are immutable
sequence unpacking : x,y,z=t # t is a tuple
A set is an unordered collection with no duplicate elements.
>>> basket = ['apple', 'orange', 'apple', 'pear', 'orange', 'banana']
>>> fruit = set(basket)               # create a set without duplicates
>>> fruit
set(['orange', 'pear', 'apple', 'banana'])
>>> 'orange' in fruit                 # fast membership testing
True


dictionary 
         a dictionary as an unordered set of key: value pairs, with the requirement that the keys are unique (within one dictionary). A pair of braces creates an empty dictionary: {}.
>>> tel = {'jack': 4098, 'sape': 4139}
>>> tel['guido'] = 4127
>>> dict([('sape', 4139), ('guido', 4127), ('jack', 4098)]) # dict() can be use to create dictionary
{'sape': 4139, 'jack': 4098, 'guido': 4127}
looping dictionary use keyword iteritems   for k,v in dict1.iteritems():
looping sequance use enumerate keyword     for i,v in seq1.enumerate():
looping multiple sequance use zip`   for s1,s2 in zip(seq1,seq2):


format functiions
str() humen readable format.
repr() interpreter readable format
rjust(n) righ allinment(justify), ljust(n) left justify, center(),  zfill(n) pading with zero
%operatoer can be use for formating like sprintf used in C/C++


File Handling
fh = open(filename, mode) mode values 'w' = write, 'r'= read, 'a'= append 'r+'=read and wirte, defult option is 'r'
data = fh.read(size) return size of data as string, if empty return end of file
line = fh.readline() return a line with new line character '\n' but last doesnot contain '\n'
fh.write(str_data) alway write string data, any other type should convert in string
fh.tell() return file poistion
fh.seek(offset, from) move file position based on from, from have values BOF=0, EOF=2, Cur=1
fh.close() close file resource open
reading
fh = open(filename,'r')
for line in fh
print line
fh.close()
writing 
fh = open(filename,'w')
fh.write(content)
fh.close()

pickle moudle
This module used for converting complext object(list, dictionary) to string and vice-versa. pickling, unpickling. for better performance use cPickle module, which is wrriten in C.
pickle.dump(obj, fh) #pickling
obj = pickle.load(fh) #unpickling

Exception Hadnling
Exception handling is working similer to C++/Java.
try:
statements
raise MyException(args, msg) raise work like throw in C++, user define exception 
except(RuntimeError, TypeError, NameError): handle multiple exception in single catch 
statements
except ValueError: handle single exception 
statements
except: handle any kind of exception like (...) in C++
statements
else if no exception occured then this block is exceuted
statements
finally: # this is use to cleanup in every circumstance
statements


Class Concept
Python is a object orianted programming language and it's support OOPS concept similer to C++
class VCalc:
def __init__(self):
print "constructor or initilization "
def add(x,y):
return x+y
def multiply(x,y):
return x*y
Class Derived(Base): # way to inharite class, think in C++ all methods of class are virtual
pass
Runtime information functions
isinstance(obj, int) will be True only if obj.__class__ is int or some class derived from int.
issubclass(bool, int) is True since bool is a subclass of int.

import os    http://docs.python.org/library/os.html#module-os
This module help to interface with operating system.
os.getlogin() # get login user name
os.getpid() # get current process id
os.getcwd() # return current working directory
os.chdir("/home/code/testing") # change current working directory
os.system("mkdir /home/viru") # run system command
os.makedirs("/home/viru/report/Trade12222010") # create directory path
os.mkdir(path) # create dirctory


import shutil
This module use to do ile and directory management tasks.
shutil.copyfile('wrong.txt', 'right.txt')
shutil.move('/home/viru/perlCode', '/home/viru/PerlCode')
shutil.copytree(src,dst) #copy recursive tree of directory from source to destination


import sys
This module is use to interact with operating system for program
sys.stderr , sys.stdout,  sys.stdin # represent respective stream
sys.exit() # exit from script
sys.call_tracing(fun, args)
sys.path, sys.platform

import re
This module use for regular expression matching, kindly follow this link whenever needed
http://docs.python.org/library/re.html#module-re
re.search(find,string)
re.match()
re.compile()
re.split()
purge(), findall(),sub()

Extra module
math: this is use for maths functions
random:  generate random number and random choice
urllib2: access internet urls
smtplib: sending mail
datetime: manipulate date and time 
zlib, gzip, bz2,zipfile,tarfile : these are use for data compression
timeit, profile, pstats : performance measurement
unittest, doctest: module for generating testing or validation modules
Important module for creating infrastructure for large applications
xmlrpc, SimpleXMLRPCServer: provide implementation for remote procedure calls(RPC) in xml protocal for wire transfer for data.
email : managing for email message
xml.dom, xml.sax: use for data interchange data formats
csv : use for csv file processing
struct : use for pack and unpack binary data, think like it create the C struct using formats
thread,threading,mutex,Queue, multiprocessing : use for multithreading
logging : logging system

default strip will remove  all space.

def read_file(filename):
        fh = open(filename,"r")
        for line in fh:
                print line.strip("\n")
        fh.close();
      
How to define main function in python
def main():
print "hello word"
if __name__ == "__main__":
    main()


How to check file exist or not
def is_file_exist(filename):
ret =0
try:
   fh = open(filename)
   fh.close()
   ret = 1
except IOError, v:
   try:
(err_code, err_msg) = v
   except:
err_code = 0
err_msg = v
   print "I/O Error: " + str(err_msg) + " (" + str(err_code) + ")"
return ret  


reading xml file 
from xml.dom import minidom
xmldoc = minidom.parse('~/viru/xmlfiles/employee.xml')
print xmldoc.firstChild.toxml()
print xmldoc.childNodes[i]
print xmldoc.getElementsByTagName('Employee')[0].firstChild.data
print xmldoc.getElementsByTagName('Employee')[0].attributes.keys()
print xmldoc.getElementsByTagName('Employee')[0].attributes.values()
print xmldoc.getElementsByTagName('Employee')[0].attributes['department'].value
print xmldoc.getElementsByTagName('Employee')[0].attributes['department'].name



Referance :
The Python Tutorial     http://docs.python.org/tutorial/
Dive Into Python      http://diveintopython.org/toc/index.html


======================================================================

======================================================================



          

Perl My View

How I used perl, what is my understanding about perl. It's not necessary everything is right, what i am writing here. I need comments from reader, so I can improve my understandings.


Command-line arguments
#argument variable
my $arg1 = $ARGV[0];
my $arg_num = $#ARGV+1;

foreach $argnum (0 .. $#ARGV) {
   print "$ARGV[$argnum]\n";
}
Check perl file syntax
perl -c perlfile.pl
Include perl module 
#!/usr/bin/perl -I module_path
Perl module format or .pm file format.
#first line :
package pakcage_name;
#last line :     this means when module load it should return true, otherwise its through compilation error.
1

#when you want to use any Module
use moudle_name;
use XML::Simple;
use Data::Dumper;

File handling
writting
open(FILEHANDLE,">$filename"); # for overwritting, for append use >>, for reading <
print FILEHANDLE $content    # no comma between filehandle and reading or writing variable
close(FILEHANDLE);
reading
open INPUT, "<$filename";
while ( <INPUT> ) {           # read line  by line
        print $_;
}

======================================================================


======================================================================
remove duplicate/get unique lines from file
my %lines;
open INFILE, "dupfile.txt"  or die "Couldn't open file: $!";
my $n=0, $nn=0;
while(<INFILE>){
        my $line = $_;
        $n =$n+1;
        if ( !exists $lines{$line}){
                $lines{$line}=$line;
                $nn = $nn+1;
        }
}
foreach $id (keys %lines){
        print STDOUT $id;
}
print STDOUT "lines ".$n." users ".$nn."\n";


subroutines format
sub mysub
{
my ($param1,$param2) = @_;  # one line get all parameter
my $param1 = $_[0];   # get parameter one based on index

$ret;   # last statement is default return statement.
}
print formated output
print <<OUT;
<Employee>
<Name>$emp_name</Name>
<EmpId>$id</EmpId>
OUT

creating XML or Hash format
my $EmployeeDetail = {'Employee'=>[{'Name'=>{'content'=>$emp_name},{'EmpId'=>{'content'=>$id}}}]};

run shell or system command

  1.  my $ouptput = `ls`; #use backtick
  2.  my $output = system("command");


XML::Simple for xml 
reading 
my $xs = new XML::Simple(KeyAttr=>[]);
my $xml_data = $xs->XMLin("$xml_file");
if (ref $xml_data->{Employee} eq "HASH"){   #single elements
}
else{     # multiple elements

my $arr_ref = $xml_data->{Employee};
my $len = scalar @$arr_ref;      # find the length of array or elements
foreach my $employee (@$arr_ref)
{
if ($employee->{Designation} eq "Manager")
{
                      print " $employee->{Name}employee is Manager";
}
}
writing

#XMLout generate xml format output from hash or array list.      
my $xml_data = $xs->XMLout( $xml_employees,
                RootName => 'Employees',
                NoSort => 1,
                KeyAttr => []
                );
#now use file operation to write the $xml_data to file
print FILEHANDLE $xml_data

Data::Dumper
#This module is use for dumping variable hash array list on STDOUT for debug purpose.
Data::Dumper($xml_data);


a simple log routine for error or debug loging
sub log_msg
{
use POSIX qw/strftime/;
my( $file, $line )= ( caller )[1,2];
my $caller_sub = ( caller(2) )[3];
if (!defined ( $caller_sub))
{
$caller_sub = "main";
}
print STDERR "$file:$line:".strftime('%D %T',localtime).":$_[0]\n";
}

getting caller filename and line number
my( $file, $line )= ( caller )[1,2];
create directory
File::Path::mkpath($dir_path) ;
check file exist or not
if( -e $filename)  # file name with path
check is this file
if(-f $filename)
string match
if($text =~m/windows/) #text contain windows
if($text =~m/windows/i) #text contain windows ignore case
if($text =~s/windows/unix/) #text substitue windows with unix
how to do simple parsing and get the output in different variables like $1,$2 
my($text) = "/home/viru/mycodes/perlCode/first.pl";
if($text =~ m/(.*\/)(.*)$/)
{
my $directory = $1;
my $filename = $2;
                print "D=$directory, F=$filename\n";
}
create a daemon routine in perl
sub daemonize {
  use POSIX;
  POSIX::setsid or die "setsid: $!";
  my $pid = fork ();
  if ($pid < 0) {
print STDERR  "die main ";
     die "fork: $!";
  } elsif ($pid) {
print STDERR "exit pid : $pid";
     exit 0;
  }
  umask 0;

  foreach (0 .. (POSIX::sysconf (&POSIX::_SC_OPEN_MAX) || 1024))
     { POSIX::close $_ }
  open (STDIN, "</dev/null");
  open (STDOUT, ">/dev/null");
  open (STDERR, ">&STDOUT");
}


Referances :
O'Reilly's Programming Perl  http://www.troubleshooters.com/codecorn/littperl/perlreg.htm
Perl Regular Expressions http://docstore.mik.ua/orelly/perl/prog3/index.htm