A tutorial for reading and writing using the input and output capabilities in Python

Read, write, and Python

The last basic step in writing a program is to read data from a file and write it to a file. After reading this article, you can add tasks to your to-do list to test the effectiveness of this skill.
Simple output

Throughout the series, 1 writes (and outputs) data directly with the print statement, which by default writes the expression as string to the screen (or console window). Listing 1 illustrates this point. Listing 1 repeats the first Python program "Hello, World!" , but made a small adjustment.
Listing 1. Simple output

>>> print "Hello World!"
Hello World!
>>> print "The total value is = $", 40.0*45.50
The total value is = $ 1820.0
>>> print "The total value = $%6.2f" % (40.0*45.50)
The total value = $1820.00
>>> myfile = file("testit.txt", 'w')
>>> print >> myfile, "Hello World!"
>>> print >> myfile, "The total value = $%6.2f" % (40.0*45.50)
>>> myfile.close()

As this example demonstrates, it is easy to write data in print statements. First, the sample outputs a simple string. Then create and output the composite string, which is created using the string formatting technique.

After that, however, things changed, unlike previous versions of the code. The next line creates the file object, passing in the name "testit.txt" and the 'w' character (to the file). The same string is then written using the modified print statement -- the two greater than signs followed by the variables that hold the file object. But this time, the data is not displayed on the screen. The natural question is: where did the data go? And what is this file object?

The first question is easy to answer. Find the testit.txt file and display its contents as shown below.

% more testit.txt 
Hello World!
The total value = $1820.00

As you can see, the data is written to the file exactly as it was written to screen 1.

Now notice the last line in listing 1, which calls the close method of the file object. This is important in Python because by default, file input and output are buffered; When the print statement is called, the data is not actually written. Instead, the data is written in batches. The easiest way to get Python to write data to a file is to explicitly call the close method.
The file object

file is the basic mechanism for interacting with files on a computer. You can use the file object to read data, write data, or add data to files, and process base 2 or text data.

The easiest way to learn about file objects is to read the help, as shown in listing 2.
Listing 2. Get help from the file object

>>> help(file)
Help on class file in module __builtin__:
class file(object)
 | file(name[, mode[, buffering]]) -> file object
 | 
 | Open a file. The mode can be 'r', 'w' or 'a' for reading (default),
 | writing or appending. The file will be created if it doesn't exist
 | when opened for writing or appending; it will be truncated when
 | opened for writing. Add a 'b' to the mode for binary files.
 | Add a '+' to the mode to allow simultaneous reading and writing.
 | If the buffering argument is given, 0 means unbuffered, 1 means line
 | buffered, and larger numbers specify the buffer size.
 | Add a 'U' to mode to open the file for input with universal newline
 | support. Any line ending in the input file will be seen as a '\n'
 | in Python. Also, a file so opened gains the attribute 'newlines';
 | the value for this attribute is one of None (no newline read yet),
 | '\r', '\n', '\r\n' or a tuple containing all the newline types seen.
 | 
 | 'U' cannot be combined with 'w' or '+' mode.
 | 
 | Note: open() is an alias for file().
 | 
 | Methods defined here:
...

As the help tool points out, using file objects is simple. Create file objects with the file constructor or the open method. open is the alias for the file constructor. The second parameter is optional and specifies how the file will be used:

      'r' (the default) means to read data from a file.       'w' means you want to write data to a file and truncate the previous content.       'a' means you want to write data to a file, but add it to the end of the current content.       'r+' means to read and write to a file (to delete all previous data).       'r+a' means to read and write to a file (add to the end of the current content).       'b' means to read and write data in base 2.

The first code listing of this article writes data to a file. Now, listing 3 shows how to read this data into the Python program and parse the contents of the file.
Listing 3. Reading data from a file

>>> myfile = open("testit.txt")
>>> myfile.read()
'Hello World!\nThe total value = $1820.00\n'
>>> str = myfile.read()
>>> print str
>>> myfile.seek(0)
>>> str = myfile.read()
>>> print str
Hello World!
The total value = $1820.00
>>> str.split()
['Hello', 'World!', 'The', 'total', 'value', '=', '$1820.00']
>>> str.split('\n')
['Hello World!', 'The total value = $1820.00', '']
>>> for line in str.split('\n'):
...   print line
... 
Hello World!
The total value = $1820.00
>>> myfile.close()

To read the data, you first create the appropriate file object -- in this case, the file object opens the testit.txt file and reads the contents using the read method. This method reads the entire file into an string and outputs the string to the console in the program. In the second call to the read method, an attempt was made to assign the value to the str variable, and an empty string was returned. This is because the first read operation reads the entire file. When you try to read the content again, you're at the end of the file, so you can't read anything.

The solution to this problem is simple: have the file object return to the beginning of the file. Going back to the beginning is done through the seek method, which takes a parameter that indicates where in the file you want to read or write from (for example, 0 represents the beginning of the file). The seek method supports more complex operations, but can be dangerous. For now, we're sticking with the simple approach.

Now that you're back at the beginning of the file, you can read the contents of the file into the string variable and parse string properly. Note that in a file, lines are distinguished by a new (or end-of-line) character. If you try to call the split method on string, it will split at a white space character, such as a space. In order for a method to split lines based on the new line character, the new line character must be explicitly specified. You can then split string and iterate through the lines of the file in the for loop.

It seems like a lot of work to just read and process 1 line from a file. Python makes simple things easy, so you might wonder if there are shortcuts available for this task. As shown in listing 4, the answer is yes.
Listing 4. Reading and parsing the lines

>>> myfile = open("testit.txt")
>>> for line in myfile.readlines():
...   print line
... 
Hello World!
The total value = $1820.00
>>> myfile.close()
>>> for line in open("testit.txt").readlines():
...   print line
... 
Hello World!
The total value = $1820.00
>>> for line in open("testit.txt"):
...   print line
... 
Hello World!
The total value = $1820.00

Listing 4 demonstrates three techniques for reading and parsing lines from a text file. First, open the file and assign it to a variable. The readlines method is then called to read the entire file into memory and break the contents into a list of string. The for loop iterates over the string list, one row at a time.

The second for loop simplifies the process slightly by using the implicit variables of the file object (that is, the variables are not explicitly created). Opening the file and reading the contents of the file are done once, producing the same effect as in the first explicit example. The last example takes a step further and simplifies and demonstrates the ability to iterate directly over file objects (note that this is a new feature of Python, so it may not work on your computer). In this example, create the implicit file object, and Python does the rest, allowing you to iterate over all the lines in the file.

Sometimes, however, you may want a better level of control when reading data from a file. In this case, you should use the readline method, as shown in listing 5.
Listing 5. Reading the data

>>> myfile = open("testit.txt")
>>> myfile.readline()
'Hello World!\n'
>>> myfile.readline()
'The total value = $1820.00\n'
>>> myfile.readline()
''
>>> myfile.seek(0)
>>> myfile.readline()
'Hello World!\n'
>>> myfile.tell()
13L
>>> myfile.readline()
'The total value = $1820.00\n'
>>> myfile.tell()
40L
>>> myfile.readline()
''
>>> myfile.tell()
40L
>>> myfile.seek(0)
>>> myfile.read(17)
'Hello World!\nThe '
>>> myfile.seek(0)
>>> myfile.readlines(23)
['Hello World!\n', 'The total value = $1820.00\n']
>>> myfile.close()

This example demonstrates how to move a file, read 1 line at a time, or explicitly move a file position indicator using the seek method. First, move around the file line using the readline method. When the end of the file is reached, the readline method returns an empty string. If you continue reading in this way after the end of the file, you will not make an error, only an empty string will be returned.

Then go back to where the file started and read another line. The tell method shows the current position in the file (after line 1 text) -- in this case, at the 13th character position. Using this knowledge, you can pass one parameter to the read or readline methods, controlling the number of characters read. For the read method, this parameter (17 in this case) is the number of characters to read from the file. The readline method, however, continues to read the specified number of characters until the end of the line. In this example, it reads the first and second lines of text.
Write data

The examples so far have focused on reading data rather than writing it. But as shown in listing 6, once you know the basics of using file objects, it's easy to write.
Listing 6. Writing data

>>> mydata = ['Hello World!', 'The total value = $1820.00']
>>> myfile = open('testit.txt', 'w')
>>> for line in mydata:
...   myfile.write(line + '\n')
... 
>>> myfile.close()
>>> myfile = open("testit.txt")
>>> myfile.read()
'Hello World!\nThe total value = $1820.00\n'
>>> myfile.close()
>>> myfile = open("testit.txt", "r+")
>>> for line in mydata:
...   myfile.write(line + '\n')
... 
>>> myfile.seek(0)
>>> myfile.read()
'Hello World!\nThe total value = $1820.00\n'
>>> myfile.close()
>>> myfile = open("testit.txt", "r+a")
>>> myfile.read()
'Hello World!\nThe total value = $1820.00\n'
>>> for line in mydata:
...   myfile.write(line + '\n')
... 
>>> myfile.seek(0)
>>> myfile.read()
'Hello World!\nThe total value = $1820.00\nHello World!\nThe total value = $1820.00\n'
>>> myfile.close()

To write data to a file, you must first create the file object. However, in this case, you must specify to write to the file using the 'w' pattern tag. In this example, write the contents of mydata list to a file, close the file, and then reopen the file so you can read the contents.

Normally, however, you want to read and write the file at the same time, so the next part of the example re-opens the file in 'r+' mode. Because you can write to a file instead of adding it, the file is truncated. First, write the contents of mydata list to the file, then reset the file pointer to the beginning of the file, and read in the contents. This example then closes the file and re-opens it in read and add mode "r+a". As shown in the sample code, the file contents are now the result of two write operations (the text is repeated).
Processing base 2 data

All of the previous examples deal with either text data or character data: write and read the character string. However, in some cases, such as when working with integers or compressed files, you need to be able to read and write base 2 data. When creating the file object, you can easily process the base 2 data with Python by adding 'b' to the file mode, as shown in listing 7.
Listing 7. Processing base 2 data

>>> myfile = open("testit.txt", "wb")
>>> for c in range(50, 70):
...   myfile.write(chr(c))
... 
>>> myfile.close()
>>> myfile = open("testit.txt")
>>> myfile.read()
'23456789:;<=>?@ABCDE'
>>> myfile.close()

In this example, create a suitable file object, and then write the base 2 characters with ASCII values from 50 to 69. Convert the integer created by the range method call into characters using the chr method. After writing all the data, close the file and reopen it for reading, again using the base 2 mode notation. Reading the file proves that the integer was not written to the file, instead, the character value was written.

You must be careful when reading and writing to base 2 data, because different platforms store base 2 data in different ways. If you have to work with base 2 data, it's best to use the appropriate object from the Python library (or from a third developer).

Read and write: the most interesting part

This article discusses how to read and write data from and to files in the Python program. Overall, the process is simple: create the appropriate file object, then read and write as needed. However, when creating an file file using write mode and writing data to the file, you must be aware of file truncation. If you need to add data to a file, you should use the add mode when creating the file object.