Monday, July 18, 2011

Your file / stdout writing Python program works on the Google App Engine

Have you ever written a Python program that writes to a file or standard out (command line) and wished you could put it on the Google App Engine without having to rewrite it? You can. Assuming your code is well organized and the proper parts are decoupled, the file writing can be faked and presented to the user in a few ways.

I've done this twice now, but I will only talk about a pure command line implementation that generates file output or writes to standard output (the other was a "web app" with a "command line" interface as well). The csvdatamix project home page discusses the overall purpose and usage of the implementation example I will use, you can head there if you want to read more about it. In general, the goal is to read data, mix or map it, and then output to a stream. This application has been on the app engine for some time now, you can see it here.

How do you do it? Use the response.out as your stream. The response.out is a StringIO object and supports write and close functions in Python, so passing it as your stream works as if it were a file or standard out.

Looking at the base implementation (command line) of the csvdatamix application, we will see the use of the file:

_mixer = CSVDataMix(hasheadings=_options.head,
   sep=_options.delimit,
   quiet=_options.quiet,
   outfile=_outputfile,
   its=_options.iterations)

I've bolded the "output file. This reference is created as:

#command line
_outputfile = sys.stdout

Or

# file writing
_outputfile = open(_options.outputfile, "wb")

The same way the sys.out is passed, the response.out can be passed in your app engine implementation:

def execGAEMix(self,sep,hasheadings,its,infile,outfile):
   _mixer = CSVDataMix(sep=sep,outfile=outfile,
      its=its,hasheadings=hasheadings)
...
mainObj.execGAEMix(_delim,_hasheadings,_iterations,
   _thefile.file,self.response.out)

Now how can this be returned to the user for consumption? In the case above, with no changes, the raw content of your application will be output to the browser. This might or might not be desired. If so, the user can simple download the page and take the data output.

Also in the app engine, you send can email. In this fashion, the data can be an attachement to the user and you can return them a proper view verses raw data.

In order to use the response.out as an attachement you need to pull it's content after the "file writing" has occurred. At this point you can clear the response.out, and then, as said, give the user a proper response.

Here is how to manipulate the response.out (see clear method):

# get and clear the response.out
_literal_string_out = self.response.out.getvalue()
self.response.clear()
...
# now write your template to the response
_path = os.path.join(os.path.dirname(__file__),
   'template_MainPage.html')
self.response.out.write(template.render(_path, _template_values))

That mostly it.

No comments:

Share on Twitter