Python RPM Specs Needing a Loving Home
@ 2010-06-22 16:37:30.688423
Filed under: Linux Code Tech Fedora Python
I found myself in need of some (currently) unpackaged python libraries and tools so I decided to do some quick packaging. If you are using these libraries (or want to) on a Fedora, RHEL or CentOS system and want to start packaging in Fedora then one of these could be an easy jumping in package. Note that they are not perfect so they still need a bit of love before getting them approved but it's (slightly) easier than starting from scratch!
django-picklefield.spec: provides an implementation of a pickled object field
python-amqplib.spec: Client library for AMQP
python-anyjson.spec: Wraps the best available JSON implementation available in a common interface
python-billiard.spec: Multiprocessing Pool Extensions
python-carrot.spec: AMQP Messaging Framework for Python
python-importlib.spec: Backport of importlib.import_module() from Python 2.7
python-celery.spec: task queue/job queue based on distributed message passing (requires all of the above)
digg it
seed it
del.icio.us
ma.gnolia
Comments: 0
python-dulwich spec file @ 2010-04-16 20:39:42
Filed under: Linux Tech Fedora Python
I was looking at Anyvc when I couldn't seem to find a dulwich package in Fedora. I threw together a spec file so I could test out anyvc (it's pretty cool by the way). I don't intend to maintain a dulwich package since I use Mecurial most of the time these days. Nothing against Git, I still like it!
If you are interested and taking the spec and running with it as a maintainer you can find it here.
digg it
seed it
del.icio.us
ma.gnolia
Comments: 0
Bad Java, BAD! No More Jars! @ 2010-03-27 23:26:42
Filed under: Code Tech Security Frustration Python
A big frustration for me is the sprawl of Jar's (java "packages") which are everywhere. These special zip files tend to be copied into other applications and then left alone. Many of these Jar's have newer releases to fix security issues, but the bundled version isn't updated. It's even worse that many Jar's don't provide enough meta information that you can be sure of who owns it. Yes, you could keep SHA1SUM's in a database like maven does, and that is better than nothing, but it's not really a fix, it's a hack.
Here is an example of metadata that came with one Jar:
How helpful! Well, we can at least see what it is by the name of the file: sqlitejdbc.jar. It still doesn't tell us what version. Let's look at another:
Better, but still isn't that helpful. In this case we get lucky as some of the info is in the file name: gettext-commons-0.9.6.jar.
It really seems like the whole 'keep metadata in your Jar' is more of an inside joke which requires a hack to try to track what jars to include (as maven does). Welcome to Jar hell. How fun. Anyway, here is another hack ...
For the heck of it I decided to write a very simple scanner. It reads the metadata from the Jar file and then tries to match it up against an online database. If it gets no results back it keeps it as 'either safe or not enough information'. If there isn't even enough data to make a call out to the database it's assumed bad and tells you the user to bother about it if that is listed, and if the database confirms vulnerabilities it's known bad. It needs a lot of work to better guess information not provided by the metadata but here is an example run (with some changes to protect the guilty)
I'm tired. I'm going to bed. I'll throw the code up somewhere tomorrow.
digg it
seed it
del.icio.us
ma.gnolia
Comments: 0
Patch For python-nmap @ 2010-03-24 16:01:12
Filed under: Code Tech Security Python
I discovered python-nmap but noted that it blocks. I threw together a quick wrapper for scanning for using in my own stuff and figured I might as well post it :-). Let's see if it goes upstream!
digg it
seed it
del.icio.us
ma.gnolia
Comments: 0
I've Lost My Mind @ 2010-03-13 23:27:00
Filed under: Code Tech Python
Heh. I like MongoDB but I don't know why I wrote this ....
digg it
seed it
del.icio.us
ma.gnolia
Comments: 0
Filed under: Linux Code Tech Fedora Python
I found myself in need of some (currently) unpackaged python libraries and tools so I decided to do some quick packaging. If you are using these libraries (or want to) on a Fedora, RHEL or CentOS system and want to start packaging in Fedora then one of these could be an easy jumping in package. Note that they are not perfect so they still need a bit of love before getting them approved but it's (slightly) easier than starting from scratch!
django-picklefield.spec: provides an implementation of a pickled object field
python-amqplib.spec: Client library for AMQP
python-anyjson.spec: Wraps the best available JSON implementation available in a common interface
python-billiard.spec: Multiprocessing Pool Extensions
python-carrot.spec: AMQP Messaging Framework for Python
python-importlib.spec: Backport of importlib.import_module() from Python 2.7
python-celery.spec: task queue/job queue based on distributed message passing (requires all of the above)
digg it
seed it
del.icio.us
ma.gnolia
Comments: 0
python-dulwich spec file @ 2010-04-16 20:39:42
Filed under: Linux Tech Fedora Python
I was looking at Anyvc when I couldn't seem to find a dulwich package in Fedora. I threw together a spec file so I could test out anyvc (it's pretty cool by the way). I don't intend to maintain a dulwich package since I use Mecurial most of the time these days. Nothing against Git, I still like it!
If you are interested and taking the spec and running with it as a maintainer you can find it here.
digg it
seed it
del.icio.us
ma.gnolia
Comments: 0
Bad Java, BAD! No More Jars! @ 2010-03-27 23:26:42
Filed under: Code Tech Security Frustration Python
A big frustration for me is the sprawl of Jar's (java "packages") which are everywhere. These special zip files tend to be copied into other applications and then left alone. Many of these Jar's have newer releases to fix security issues, but the bundled version isn't updated. It's even worse that many Jar's don't provide enough meta information that you can be sure of who owns it. Yes, you could keep SHA1SUM's in a database like maven does, and that is better than nothing, but it's not really a fix, it's a hack.
Here is an example of metadata that came with one Jar:
Manifest-Version: 1.0 Created-By: 1.6.0_10 (Sun Microsystems Inc.)
How helpful! Well, we can at least see what it is by the name of the file: sqlitejdbc.jar. It still doesn't tell us what version. Let's look at another:
Manifest-Version: 1.0 Archiver-Version: Plexus Archiver Created-By: Apache Maven Built-By: fberger Build-Jdk: 1.6.0_06
Better, but still isn't that helpful. In this case we get lucky as some of the info is in the file name: gettext-commons-0.9.6.jar.
It really seems like the whole 'keep metadata in your Jar' is more of an inside joke which requires a hack to try to track what jars to include (as maven does). Welcome to Jar hell. How fun. Anyway, here is another hack ...
For the heck of it I decided to write a very simple scanner. It reads the metadata from the Jar file and then tries to match it up against an online database. If it gets no results back it keeps it as 'either safe or not enough information'. If there isn't even enough data to make a call out to the database it's assumed bad and tells you the user to bother about it if that is listed, and if the database confirms vulnerabilities it's known bad. It needs a lot of work to better guess information not provided by the metadata but here is an example run (with some changes to protect the guilty)
$ python jarscanner.py *jar
WARNING:root:apache-mime4j-0.6.jar is the latest secure version or not enough info
WARNING:root:commons-codec-1.3.jar is the latest secure version or not enough info
WARNING:root:commons-logging-1.1.1.jar is the latest secure version or not enough info
WARNING:root:hsqldb.jar is the latest secure version or not enough info
WARNING:root:httpclient-4.0.jar is the latest secure version or not enough info
WARNING:root:httpcore-4.0.1.jar is the latest secure version or not enough info
WARNING:root:httpmime-4.0.jar is the latest secure version or not enough info
INFO:root:jetty-6.1.7.jar found 15 vulns
INFO:root:jetty-util-6.1.7.jar found 15 vulns
WARNING:root:servlet-api-2.5-6.1.7.jar is the latest secure version or not enough info
WARNING:root:The following jars are known to be bad ...
WARNING:root:jetty-6.1.7.jar
WARNING:root:jetty-util-6.1.7.jar
CRITICAL:root:Sorry, but a number of jars are crap and don't provide enough information.
These should be assumed bad!!!
CRITICAL:root:bdiff.jar
CRITICAL:root:fast-md5.jar (go bug dragonlz about it)
CRITICAL:root:gettext-commons-0.9.6.jar (go bug fberger about it)
CRITICAL:root:jcip-annotations.jar
CRITICAL:root:linuxfolderwatcher.jar
CRITICAL:root:messages.jar
CRITICAL:root:snakeyaml-1.5.jar (go bug somov about it)
CRITICAL:root:sqlitejdbc.jar
CRITICAL:root:stringtree-json-2.0.9.jar
CRITICAL:root:swt.jar
CRITICAL:root:unixapi.jar
CRITICAL:root:XXXXXXXXXX.jar
CRITICAL:root:XXXXXXXXXX.jar
I'm tired. I'm going to bed. I'll throw the code up somewhere tomorrow.
digg it
seed it
del.icio.us
ma.gnolia
Comments: 0
Patch For python-nmap @ 2010-03-24 16:01:12
Filed under: Code Tech Security Python
I discovered python-nmap but noted that it blocks. I threw together a quick wrapper for scanning for using in my own stuff and figured I might as well post it :-). Let's see if it goes upstream!
#!/usr/bin/env python
# By Steve 'Ashcrow' Milner
import nmap
try:
from multiprocessing import Process, Queue
except ImportError:
# For pre 2.6 releases
from threading import Thread as Process
from Queue import Queue
class NmapProcess(object):
"""
Process scans in a non-blocking way.
"""
def __init__(self, host):
"""
Creates an instance.
:Parameters:
- `host`: host to scan
"""
self.__host = host
self.__queue = Queue()
self.__nm = nmap.PortScanner()
self.__process = None
def scan(self, ports=None, arguments='-sV', callback=None):
"""
Starts the scan taking the same input as nmap scan except the host.
:Parameters:
- `ports`: ports to scan
- `arguments` arguments for nmap
- `callback`: optional callback to execute when results are available
"""
def scan_process(ports, arguments, callback=None):
self.__nm.scan(self.__host, ports, arguments)
self.__queue.put(self.__nm)
if callback and callable(callback):
callback(self.__nm)
self.__process = Process(
target=scan_process, args=(ports, arguments, callback))
self.__process.daemon = True
self.__process.start()
def __get_results(self):
"""
Returns results if they are available, else None.
"""
if self.__queue.empty():
return None
return self.__queue.get()[self.__host]
def __del__(self):
"""
Clean up.
"""
self.__process.join()
# Properties
results = property(__get_results)
results_available = property(lambda s: not s.__process.is_alive())
if __name__ == '__main__':
n = NmapProcess('127.0.0.1')
def p(s):
print("From Callback: " + str(s['127.0.0.1']))
n.scan(arguments="-sV -T4 -A", callback=p)
# Do stuff here ...
from time import sleep
while not n.results_available:
print("Waiting ...")
sleep(3)
print("From property: " + str(n.results))
digg it
seed it
del.icio.us
ma.gnolia
Comments: 0
I've Lost My Mind @ 2010-03-13 23:27:00
Filed under: Code Tech Python
Heh. I like MongoDB but I don't know why I wrote this ....
class _MongoSaveable(object):
"""
Simple mongodb saveable object. Saves all attributes.
You can force saves using a shortcut like so:
>>> save = lambda s: s.__mongo_save__(COLLECTION)
>>> save(my_instance)
"""
_id = None
_dirty = False
def __setattr__(self, name, value):
"""
TRY TO CATCH ME RIDN DIRTY *money money money*
:Parameters:
- `name`: key name
- `value`: data to store
"""
self.__dict__['_dirty'] = True
self.__dict__[name] = value
def __mongo_save__(self, collection):
"""
Insert or Update data.
:Parameters:
- `collection`: the collection to associate with
"""
if not self._dirty:
return None
result = {}
for x in dir(self):
if (not x.startswith('_') and
not callable(getattr(self, x)) and
x not in ("id", "dirty")):
result[x] = getattr(self, x)
if self._id:
collection.save(result)
else:
saved_data = collection.find_one(result)
if saved_data:
self._id = saved_data["_id"]
else:
self._id = collection.insert(result)
self._dirty = False
dirty = property(lambda s: s._dirty)
id = property(lambda s: s._id)
# PSUEDO EXAMPLE!!!!
import pymongo
from myapp import Example
# Connection stuff
DB = pymongo.Connection().my_database
COLLECTION = DB.my_collection
# Save shortcut
save = lambda s: s.__mongo_save__(COLLECTION)
class DataExample(Example, _MongoSaveable):
def __init__(self, weight):
self.weight = weight
def calculate_status(self):
if self.weight > 175:
self.status = "lose wight"
else:
self.status = "doin fine"
ex = DataExample(150)
ex.calculate_status()
save(ex)
print ex.id # 4b9c6398bdcf5f4053000000
ex.goal_weight = 160
save(ex)
print ex.id # 4b9c6398bdcf5f4053000000
digg it
seed it
del.icio.us
ma.gnolia
Comments: 0

