Parallel Python Example
@ 2008-11-14 22:05:40
Filed under: Code Python Tech
Parallel Python is one cool library. Here is an example of running jobs on a 'pool' of job servers (in this case just one ... but still) print out the results as they become available.
Here is output of a few runs ...
digg it
seed it
del.icio.us
ma.gnolia
Pep8 Patches For Loggerhead @ 2008-10-24 14:17:35
Filed under: Code Python Tech
They just got approved and merged. Yay open source!
digg it
seed it
del.icio.us
ma.gnolia
Director On Different Runtimes @ 2008-10-19 18:55:07
Filed under: Code director Python Tech
Director works with CPython ... it's what I develop directly on top of and has the biggest user base. When I released director 1.1.0 I ran the tests using the upcoming Jython 2.5 release and it passed ... You can use director 1.1.0 with Jython 2.5 (and probably later). I have not had the same luck with IronPython yet ... I've actually found IronPython can do some os module items via it's nt module. But after finding that out I've found it does not have the warnings, optparse, inspect or types modules which are core to director. I've put some thought into making an abstraction class that can use some of the .net libraries that might be able to do what those modules did but it's a bit more work than I would have expected to do ... I've also been told that I could ship those modules with my library for IronPython users ... but that seems dirty! For now I'll do it this way ... if you use IronPython and want to use director on it shoot me an email (steve m at gnu linux dot net) and let me know ... feel free to pass on any ideas on getting it to work :-)
digg it
seed it
del.icio.us
ma.gnolia
Director 1.1.0 Released @ 2008-10-16 17:53:41
Filed under: Code director Fedora Python Tech
Subject says it all, checkout the info at the Fedora Hosted Project, Python package docs, Pypi and Ohloh. Patches always welcome :-).
digg it
seed it
del.icio.us
ma.gnolia
Director May Work Find With Jython Without Changes @ 2008-10-12 16:35:13
Filed under: Code Python Tech
Though it isn't as fast as python on it's own ...
digg it
seed it
del.icio.us
ma.gnolia
Filed under: Code Python Tech
Parallel Python is one cool library. Here is an example of running jobs on a 'pool' of job servers (in this case just one ... but still) print out the results as they become available.
#!/usr/bin/env python
"""
Example script that executes jobs via a pool of job servers and prints the
results when they become ready ... not in any predetermined order.
"""
import pp
from random import randint
def sleep(name, wait):
"""
Example function to run as a job.
"""
time.sleep(wait)
return "%s (%s)" %(name, wait)
def main():
"""
Main function.
"""
# Pool of ppservers to use as a 'server'
job_server = pp.Server(ppservers=('127.0.0.1', ))
# Array of all submitted jobs
jobs = []
# Function Params Libs/Modules
# v v v
jobs.append(job_server.submit(sleep, ('f1', randint(3,15)), (), ('time',)))
jobs.append(job_server.submit(sleep, ('f2', randint(3,15)), (), ('time',)))
jobs.append(job_server.submit(sleep, ('f3', randint(3,15)), (), ('time',)))
jobs.append(job_server.submit(sleep, ('f4', randint(3,15)), (), ('time',)))
# Loop
while True:
jobs_len = len(jobs)
# For each job ...
for x in range(jobs_len):
# See if it's finished
if jobs[x].finished == True:
# If so, print the result, delete the job and loop again
print jobs[x]()
del jobs[x]
break
# All out of jobs? Exit!
if jobs_len == 0:
raise SystemExit(0)
if __name__ == '__main__':
main()
Here is output of a few runs ...
[steve@tachikoman ~]$ python ppexample.py
f4 (3)
f1 (4)
f3 (6)
f2 (11)
[steve@tachikoman ~]$ python ppexample.py
f4 (3)
f3 (3)
f1 (4)
f2 (9)
[steve@tachikoman ~]$ python ppexamples.py
f1 (7)
f4 (8)
f2 (13)
f3 (14)
Pep8 Patches For Loggerhead @ 2008-10-24 14:17:35
Filed under: Code Python Tech
They just got approved and merged. Yay open source!
digg it
seed it
del.icio.us
ma.gnolia
Director On Different Runtimes @ 2008-10-19 18:55:07
Filed under: Code director Python Tech
Director works with CPython ... it's what I develop directly on top of and has the biggest user base. When I released director 1.1.0 I ran the tests using the upcoming Jython 2.5 release and it passed ... You can use director 1.1.0 with Jython 2.5 (and probably later). I have not had the same luck with IronPython yet ... I've actually found IronPython can do some os module items via it's nt module. But after finding that out I've found it does not have the warnings, optparse, inspect or types modules which are core to director. I've put some thought into making an abstraction class that can use some of the .net libraries that might be able to do what those modules did but it's a bit more work than I would have expected to do ... I've also been told that I could ship those modules with my library for IronPython users ... but that seems dirty! For now I'll do it this way ... if you use IronPython and want to use director on it shoot me an email (steve m at gnu linux dot net) and let me know ... feel free to pass on any ideas on getting it to work :-)
digg it
seed it
del.icio.us
ma.gnolia
Director 1.1.0 Released @ 2008-10-16 17:53:41
Filed under: Code director Fedora Python Tech
Subject says it all, checkout the info at the Fedora Hosted Project, Python package docs, Pypi and Ohloh. Patches always welcome :-).
digg it
seed it
del.icio.us
ma.gnolia
Director May Work Find With Jython Without Changes @ 2008-10-12 16:35:13
Filed under: Code Python Tech
[steve@tachikoman director]$ date; ~/jython2.5a3/jython setup.py build; \
~/jython2.5a3/jython setup.py test; date
Sun Oct 12 16:32:49 EDT 2008
running build
running build_py
creating build
creating build/lib
creating build/lib/director
copying src/director/__init__.py -> build/lib/director
copying src/director/decorators.py -> build/lib/director
copying src/director/filter.py -> build/lib/director
running test
test_parse_options (tests.test_director.ActionRunnerTests) ... ok
test_run_code (tests.test_director.ActionRunnerTests) ... ok
test_run_with_filter (tests.test_director.ActionRunnerTests) ... ok
test__action_help (tests.test_director.ActionTests) ... ok
test__list_verbs (tests.test_director.ActionTests) ... ok
test_description (tests.test_director.ActionTests) ... ok
test_help (tests.test_director.ActionTests) ... ok
test_filter (tests.test_filter.ExceptionFilterTests) ... ok
test_execute_filters (tests.test_filter.FilterTests) ... ok
test_register_filter (tests.test_filter.FilterTests) ... ok
test_general_help (tests.test_decorators.ActionTests) ... ok
test_simple_help (tests.test_decorators.ActionTests) ... ok
----------------------------------------------------------------------
Ran 12 tests in 0.235s
OK
Sun Oct 12 16:32:55 EDT 2008
[steve@tachikoman director]$
Though it isn't as fast as python on it's own ...
[steve@tachikoman director]$ rm -rf build/
[steve@tachikoman director]$ date; ./setup.py build; ./setup.py test; date
Sun Oct 12 16:34:30 EDT 2008
running build
running build_py
creating build
creating build/lib
creating build/lib/director
copying src/director/__init__.py -> build/lib/director
copying src/director/decorators.py -> build/lib/director
copying src/director/filter.py -> build/lib/director
running test
test_parse_options (tests.test_director.ActionRunnerTests) ... ok
test_run_code (tests.test_director.ActionRunnerTests) ... ok
test_run_with_filter (tests.test_director.ActionRunnerTests) ... ok
test__action_help (tests.test_director.ActionTests) ... ok
test__list_verbs (tests.test_director.ActionTests) ... ok
test_description (tests.test_director.ActionTests) ... ok
test_help (tests.test_director.ActionTests) ... ok
test_filter (tests.test_filter.ExceptionFilterTests) ... ok
test_execute_filters (tests.test_filter.FilterTests) ... ok
test_register_filter (tests.test_filter.FilterTests) ... ok
test_general_help (tests.test_decorators.ActionTests) ... ok
test_simple_help (tests.test_decorators.ActionTests) ... ok
----------------------------------------------------------------------
Ran 12 tests in 0.009s
OK
Sun Oct 12 16:34:30 EDT 2008
