stevemilner.org Blog

Python RPM Specs Needing a Loving Home @ 2010-06-22 16:37:30.688423
Filed under: Linux Code Tech Fedora Python
I found myself in need of some (currently) unpackaged python libraries and tools so I decided to do some quick packaging. If you are using these libraries (or want to) on a Fedora, RHEL or CentOS system and want to start packaging in Fedora then one of these could be an easy jumping in package. Note that they are not perfect so they still need a bit of love before getting them approved but it's (slightly) easier than starting from scratch!

django-picklefield.spec: provides an implementation of a pickled object field
python-amqplib.spec: Client library for AMQP
python-anyjson.spec: Wraps the best available JSON implementation available in a common interface
python-billiard.spec: Multiprocessing Pool Extensions
python-carrot.spec: AMQP Messaging Framework for Python
python-importlib.spec: Backport of importlib.import_module() from Python 2.7
python-celery.spec: task queue/job queue based on distributed message passing (requires all of the above)

digg it

seed it

del.icio.us

ma.gnolia
Comments: 0 Tags: Linux Code Tech Fedora Python

Changelog Format @ 2010-06-14 22:06:54.198506
Filed under: Code Tech Frustration Fedora
This has been bugging me fore a while. Many projects and products have a changelog. It's great! We can see in a file what changes have been made from release 1 to release 2. Wouldn't it be great to watch an upstream changelog file and use something like Buildbot that would trigger a build on a new release. Then my brain started working. It told me that humans are much better at parsing information provided in different textual formats or markups. Let me explain ....

Let's take a look at the victims project. Even though victims doesn't have a special changelog, we will follow the general scm changelog information. Victims has a changelog like so:

2010-05-20  Steve 'Ashcrow' Milner  

        * setup.py:
        added archivers module to the setup script
        [4cd8f0133b44] [tip]

2010-05-18  Steve 'Ashcrow' Milner  

        * README, src/victims/__init__.py, src/victims/archivers/__init__.py:
        rpm is now listed as a useable archive closing #8
        [e71ad437f9f4]

Based off this information we can easily create a parser! We care about the date, author/email, description and the release (tag). Through the magic of a little bit of regex the following works decent enough ...

(\d{4}-\d{2}-\d{2})  (.*)  <(.*)>\n\n.*:\n[ ]*(.*)\n[ ]*(.*)

Now we can parse changelogs! Yay! Oh, but then our brain explodes in fear since this is not the only project out there. Surely everyone uses the same format! Let's use nmap as a second project example.

# Nmap Changelog ($Id: CHANGELOG 18109 2010-06-14 18:48:07Z drazen $); -*-text-*-

o [NSE] Added additional vulnerability checks to smb-check-vulns.nse. These checks
  are intrusive and have MS06-025, MS07-029 designations.
  
o [NSE] Added dns-cache-snoop.nse by Eugene Alexeev. This script does
  cache snooping by either sending non-recursive queries or by measuring
  response times.

Well that isn't so bad! With a little regex we could ... wait ... if I have to do this twice with two different projects am I going to need to do this many, many more times before I create Skynet^H^H^H^H^H^Han uber parser smart enough to figure out what accent, dialect, markup, etc.. a changelog may be in? It sure seems that way!

This is when another thought entered by brain (TWO IN ONE DAY!!!): Surely someone else has thought of this. There must be a commonly used format that shares this information for easy inclusion. As it turns out, I could only find one format for this and it doesn't exactly match. The project I'm talking about is doap. While the project does seem interesting, it seems to focus more so on information about a project and it's services and not so much about project releases and changes that have happened between those releases.

Long story long .... am I out of luck? Is there not a format in the works to deal with release information such as this in an open way? If there really isn't, is anyone interested in creating a format? It seems to me that this would be quite useful for package maintainers, system administrators and developers. Hit me up on identi.ca or twitter if you know or a format or want to chat about what one would look like.

digg it

seed it

del.icio.us

ma.gnolia
Comments: 0 Tags: Code Tech Frustration Fedora

python-dulwich spec file @ 2010-04-16 20:39:42
Filed under: Linux Tech Fedora Python
I was looking at Anyvc when I couldn't seem to find a dulwich package in Fedora. I threw together a spec file so I could test out anyvc (it's pretty cool by the way). I don't intend to maintain a dulwich package since I use Mecurial most of the time these days. Nothing against Git, I still like it!

If you are interested and taking the spec and running with it as a maintainer you can find it here.

digg it

seed it

del.icio.us

ma.gnolia
Comments: 0 Tags: Linux Tech Fedora Python

Mongoose Web Server Spec @ 2010-03-10 20:19:33
Filed under: Linux Code Tech Fedora Python
I happened to take a look at this and see there wasn't a package for it. It's one of those items that I don't have time to keep up with but I think would be a nice package to have in Fedora. If anyone wants to pick up with this and run with it be my guest!

The package includes a subpackage for devel (a single header file) and a subpackage for the Python bindings. There is also a patch to get the Python code to find the shared object.

Have fun!

Patch: mongoose-site-location.patch

--- bindings/python/mongoose.py	2010-03-10 20:07:53.735407453 -0500
+++ bindings/python/mongoose.py	2010-03-10 20:09:08.935760549 -0500
@@ -110,7 +110,8 @@
 
 	def __init__(self, **kwargs):
 		dll_extension = os.name == 'nt' and 'dll' or 'so'
-		self.dll = ctypes.CDLL('_mongoose.%s' % dll_extension)
+		from distutils.sysconfig import get_python_lib
+		self.dll = ctypes.CDLL(get_python_lib(1) + '/_mongoose.%s' % dll_extension)
 		start = self.dll.mg_start
 		self.ctx = ctypes.c_voidp(self.dll.mg_start()).value
 		self.version = ctypes.c_char_p(self.dll.mg_version()).value

Spec:

# sitearch for others (remove the unneeded one)
%{!?python_sitearch: %global python_sitearch %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib(1))")}


Name:           mongoose
Version:        2.8
Release:        1%{?dist}
Summary:        Simple and easy to use web server

Group:          System Environment/Daemons
License:        MIT
URL:            http://code.google.com/p/mongoose/
Source0:        http://mongoose.googlecode.com/files/%{name}-%{version}.tgz
Patch0:         mongoose-site-location.patch
BuildRoot:      %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)


%description
Mongoose is an easy to use web server. It can be embedded into
existing application to provide a web interface to it.


%package python
Summary:       Python bindings for the mongoose web server
Requires:      mongoose


%description python
Mongoose is an easy to use web server. It can be embedded into
existing application to provide a web interface to it. This
package includes the bindings for the Python programming language.


%package devel
Summary:       Development files for the mongoose web server
BuildArch:     noarch


%description devel
Mongoose is an easy to use web server. It can be embedded into
existing application to provide a web interface to it. This
package includes the development files.


%prep
%setup -qn %{name}
%patch0


%build
make %{?_smp_mflags} linux


%install
rm -rf $RPM_BUILD_ROOT
# Install the base
mkdir -p $RPM_BUILD_ROOT/%{_bindir}
mkdir -p $RPM_BUILD_ROOT/%{_mandir}/man1/
cp %{name} $RPM_BUILD_ROOT/%{_bindir}
cp %{name}.1 $RPM_BUILD_ROOT/%{_mandir}/man1/

# Install the python bindings
mkdir -p $RPM_BUILD_ROOT/%{python_sitearch}
cp bindings/python/mongoose.py _%{name}.so $RPM_BUILD_ROOT/%{python_sitearch}

# Install the development files
mkdir -p $RPM_BUILD_ROOT/%{_includedir}/%{name}/
cp %{name}.h $RPM_BUILD_ROOT/%{_includedir}/%{name}/


%clean
rm -rf $RPM_BUILD_ROOT


%files
%defattr(-,root,root,-)
%{_bindir}/%{name}
%{_mandir}/man1/%{name}.1.gz


%files python
%defattr(-,root,root,-)
%{python_sitearch}/_%{name}.so
%{python_sitearch}/%{name}.py*


%files devel
%defattr(-,root,root,-)
%{_includedir}/%{name}/


%changelog
* Wed Mar 10 2010 Steve 'Ashcrow' Milner <me@stevemilner.org> 2.8-1
- Initial spec

digg it

seed it

del.icio.us

ma.gnolia
Comments: 0 Tags: Linux Code Tech Fedora Python

Facebook's tornado rpm spec @ 2009-09-12 15:47:49
Filed under: Code Tech Fedora Python
Not sure if I'm going to push this up as I don't see myself using it (and thus maintain the package properly) so if you want to run with it, go ahead :-).

Edit: Some feeds were reporting funny results with the rendered code tags ... changed to pre.

[steve@mobileoppressionpalace SPECS]$ cat tornado.spec                                                                   
# sitelib for noarch packages                                                                                            
%{!?python_sitelib: %global python_sitelib %(%{__python} -c \
"from distutils.sysconfig import get_python_lib; print get_python_lib()")}                                                                                                                        

Name:           tornado
Version:        0.1    
Release:        1%{?dist}
Summary:        An open source version of the scalable, non-blocking web server

Group:          Development/Languages
License:        ASL 2.0
URL:            http://www.tornadoweb.org/
Source0:        http://www.tornadoweb.org/static/%{name}-%{version}.tar.gz
Patch0:         tornado-no-hashbang-for-non-exec-scripts.patch
BuildRoot:      %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)

BuildArch:      noarch
BuildRequires:  python-devel
Requires:       python-pycurl, python-simplejson


%description
Tornado is an open source version of the scalable, non-blocking web server
and tools that power FriendFeed. The FriendFeed application is written using
a web framework that looks a bit like web.py or Google's webapp, but with
additional tools and optimizations to take advantage of the underlying
non-blocking infrastructure.

%prep
%setup -q
%patch0 -p 1


%build
%{__python} setup.py build


%install
rm -rf $RPM_BUILD_ROOT
%{__python} setup.py install -O1 --skip-build --root $RPM_BUILD_ROOT


%clean
rm -rf $RPM_BUILD_ROOT


%files
%defattr(-,root,root,-)
%doc README
# For noarch packages: sitelib
%{python_sitelib}/*


%changelog
* Sat Sep 12 2009 Steve 'Ashcrow' Milner  0.1-1
- Initial package

[steve@mobileoppressionpalace SPECS]$ cat ../SOURCES/tornado-no-hashbang-for-non-exec-scripts.patch 
diff -ur tornado-0.1/tornado/auth.py tornado-0.1.mine/tornado/auth.py                               
--- tornado-0.1/tornado/auth.py 2009-09-10 13:32:56.000000000 -0400                                 
+++ tornado-0.1.mine/tornado/auth.py    2009-09-12 14:01:08.716733153 -0400                         
@@ -1,5 +1,3 @@                                                                                     
-#!/usr/bin/env python                                                                              
-#                                                                                                  
 # Copyright 2009 Facebook                                                                          
 #                                                                                                  
 # Licensed under the Apache License, Version 2.0 (the "License"); you may                          
diff -ur tornado-0.1/tornado/database.py tornado-0.1.mine/tornado/database.py                       
--- tornado-0.1/tornado/database.py     2009-09-10 13:32:56.000000000 -0400                         
+++ tornado-0.1.mine/tornado/database.py        2009-09-12 14:02:40.012605235 -0400                 
@@ -1,5 +1,3 @@                                                                                     
-#!/usr/bin/env python                                                                              
-#                                                                                                  
 # Copyright 2009 Facebook                                                                          
 #                                                                                                  
 # Licensed under the Apache License, Version 2.0 (the "License"); you may                          
diff -ur tornado-0.1/tornado/escape.py tornado-0.1.mine/tornado/escape.py                           
--- tornado-0.1/tornado/escape.py       2009-09-10 13:32:56.000000000 -0400                         
+++ tornado-0.1.mine/tornado/escape.py  2009-09-12 14:02:48.993692391 -0400                         
@@ -1,5 +1,3 @@                                                                                     
-#!/usr/bin/env python                                                                              
-#                                                                                                  
 # Copyright 2009 Facebook                                                                          
 #                                                                                                  
 # Licensed under the Apache License, Version 2.0 (the "License"); you may                          
diff -ur tornado-0.1/tornado/httpclient.py tornado-0.1.mine/tornado/httpclient.py                   
--- tornado-0.1/tornado/httpclient.py   2009-09-10 13:32:56.000000000 -0400                         
+++ tornado-0.1.mine/tornado/httpclient.py      2009-09-12 14:03:04.558617851 -0400                 
@@ -1,5 +1,3 @@                                                                                     
-#!/usr/bin/env python                                                                              
-#                                                                                                  
 # Copyright 2009 Facebook                                                                          
 #                                                                                                  
 # Licensed under the Apache License, Version 2.0 (the "License"); you may                          
diff -ur tornado-0.1/tornado/httpserver.py tornado-0.1.mine/tornado/httpserver.py                   
--- tornado-0.1/tornado/httpserver.py   2009-09-10 13:32:56.000000000 -0400                         
+++ tornado-0.1.mine/tornado/httpserver.py      2009-09-12 14:03:15.499707010 -0400                 
@@ -1,5 +1,3 @@                                                                                     
-#!/usr/bin/env python                                                                              
-#                                                                                                  
 # Copyright 2009 Facebook                                                                          
 #                                                                                                  
 # Licensed under the Apache License, Version 2.0 (the "License"); you may                          
diff -ur tornado-0.1/tornado/__init__.py tornado-0.1.mine/tornado/__init__.py                       
--- tornado-0.1/tornado/__init__.py     2009-09-10 13:32:56.000000000 -0400                         
+++ tornado-0.1.mine/tornado/__init__.py        2009-09-12 14:03:24.101684722 -0400                 
@@ -1,5 +1,3 @@                                                                                     
-#!/usr/bin/env python                                                                              
-#                                                                                                  
 # Copyright 2009 Facebook                                                                          
 #                                                                                                  
 # Licensed under the Apache License, Version 2.0 (the "License"); you may                          
diff -ur tornado-0.1/tornado/ioloop.py tornado-0.1.mine/tornado/ioloop.py                           
--- tornado-0.1/tornado/ioloop.py       2009-09-10 13:32:56.000000000 -0400                         
+++ tornado-0.1.mine/tornado/ioloop.py  2009-09-12 14:03:43.310690548 -0400                         
@@ -1,5 +1,3 @@                                                                                     
-#!/usr/bin/env python                                                                              
-#                                                                                                  
 # Copyright 2009 Facebook                                                                          
 #                                                                                                  
 # Licensed under the Apache License, Version 2.0 (the "License"); you may                          
diff -ur tornado-0.1/tornado/iostream.py tornado-0.1.mine/tornado/iostream.py                       
--- tornado-0.1/tornado/iostream.py     2009-09-10 13:32:56.000000000 -0400                         
+++ tornado-0.1.mine/tornado/iostream.py        2009-09-12 14:03:49.922692867 -0400                 
@@ -1,5 +1,3 @@                                                                                     
-#!/usr/bin/env python                                                                              
-#                                                                                                  
 # Copyright 2009 Facebook                                                                          
 #                                                                                                  
 # Licensed under the Apache License, Version 2.0 (the "License"); you may                          
diff -ur tornado-0.1/tornado/locale.py tornado-0.1.mine/tornado/locale.py                           
--- tornado-0.1/tornado/locale.py       2009-09-10 13:32:56.000000000 -0400                         
+++ tornado-0.1.mine/tornado/locale.py  2009-09-12 14:03:56.554607130 -0400                         
@@ -1,5 +1,3 @@                                                                                     
-#!/usr/bin/env python                                                                              
-#                                                                                                  
 # Copyright 2009 Facebook
 #
 # Licensed under the Apache License, Version 2.0 (the "License"); you may
diff -ur tornado-0.1/tornado/options.py tornado-0.1.mine/tornado/options.py
--- tornado-0.1/tornado/options.py      2009-09-10 13:32:56.000000000 -0400
+++ tornado-0.1.mine/tornado/options.py 2009-09-12 14:04:11.897690510 -0400
@@ -1,5 +1,3 @@
-#!/usr/bin/env python
-#
 # Copyright 2009 Facebook
 #
 # Licensed under the Apache License, Version 2.0 (the "License"); you may
diff -ur tornado-0.1/tornado/s3server.py tornado-0.1.mine/tornado/s3server.py
--- tornado-0.1/tornado/s3server.py     2009-09-10 13:32:56.000000000 -0400
+++ tornado-0.1.mine/tornado/s3server.py        2009-09-12 14:04:18.419607955 -0400
@@ -1,5 +1,3 @@
-#!/usr/bin/env python
-#
 # Copyright 2009 Facebook
 #
 # Licensed under the Apache License, Version 2.0 (the "License"); you may
diff -ur tornado-0.1/tornado/template.py tornado-0.1.mine/tornado/template.py
--- tornado-0.1/tornado/template.py     2009-09-10 13:32:56.000000000 -0400
+++ tornado-0.1.mine/tornado/template.py        2009-09-12 14:04:25.778730891 -0400
@@ -1,5 +1,3 @@
-#!/usr/bin/env python
-#
 # Copyright 2009 Facebook
 #
 # Licensed under the Apache License, Version 2.0 (the "License"); you may
diff -ur tornado-0.1/tornado/web.py tornado-0.1.mine/tornado/web.py
--- tornado-0.1/tornado/web.py  2009-09-10 13:32:56.000000000 -0400
+++ tornado-0.1.mine/tornado/web.py     2009-09-12 14:04:32.843607704 -0400
@@ -1,5 +1,3 @@
-#!/usr/bin/env python
-#
 # Copyright 2009 Facebook
 #
 # Licensed under the Apache License, Version 2.0 (the "License"); you may
diff -ur tornado-0.1/tornado/wsgi.py tornado-0.1.mine/tornado/wsgi.py
--- tornado-0.1/tornado/wsgi.py 2009-09-10 13:32:56.000000000 -0400
+++ tornado-0.1.mine/tornado/wsgi.py    2009-09-12 14:04:39.124604938 -0400
@@ -1,5 +1,3 @@
-#!/usr/bin/env python
-#
 # Copyright 2009 Facebook
 #
 # Licensed under the Apache License, Version 2.0 (the "License"); you may

digg it

seed it

del.icio.us

ma.gnolia
Comments: 0 Tags: Code Tech Fedora Python