Home

Changelog Format @ 2010-06-14 22:06:54.198506
Filed under: Code  Tech  Frustration  Fedora 
This has been bugging me fore a while. Many projects and products have a changelog. It's great! We can see in a file what changes have been made from release 1 to release 2. Wouldn't it be great to watch an upstream changelog file and use something like Buildbot that would trigger a build on a new release. Then my brain started working. It told me that humans are much better at parsing information provided in different textual formats or markups. Let me explain ....

Let's take a look at the victims project. Even though victims doesn't have a special changelog, we will follow the general scm changelog information. Victims has a changelog like so:

2010-05-20  Steve 'Ashcrow' Milner  

        * setup.py:
        added archivers module to the setup script
        [4cd8f0133b44] [tip]

2010-05-18  Steve 'Ashcrow' Milner  

        * README, src/victims/__init__.py, src/victims/archivers/__init__.py:
        rpm is now listed as a useable archive closing #8
        [e71ad437f9f4]


Based off this information we can easily create a parser! We care about the date, author/email, description and the release (tag). Through the magic of a little bit of regex the following works decent enough ... 

(\d{4}-\d{2}-\d{2})  (.*)  <(.*)>\n\n.*:\n[ ]*(.*)\n[ ]*(.*)


Now we can parse changelogs! Yay! Oh, but then our brain explodes in fear since this is not the only project out there. Surely everyone uses the same format! Let's use nmap as a second project example.

# Nmap Changelog ($Id: CHANGELOG 18109 2010-06-14 18:48:07Z drazen $); -*-text-*-

o [NSE] Added additional vulnerability checks to smb-check-vulns.nse. These checks
  are intrusive and have MS06-025, MS07-029 designations.
  
o [NSE] Added dns-cache-snoop.nse by Eugene Alexeev. This script does
  cache snooping by either sending non-recursive queries or by measuring
  response times.


Well that isn't so bad! With a little regex we could ... wait ... if I have to do this twice with two different projects am I going to need to do this many, many more times before I create Skynet^H^H^H^H^H^Han uber parser smart enough to figure out what accent, dialect, markup, etc.. a changelog may be in? It sure seems that way!

This is when another thought entered by brain (TWO IN ONE DAY!!!): Surely someone else has thought of this. There must be a commonly used format that shares this information for easy inclusion. As it turns out, I could only find one format for this and it doesn't exactly match. The project I'm talking about is doap. While the project does seem interesting, it seems to focus more so on information about a project and it's services and not so much about project releases and changes that have happened between those releases.

Long story long .... am I out of luck? Is there not a format in the works to deal with release information such as this in an open way? If there really isn't, is anyone interested in creating a format? It seems to me that this would be quite useful for package maintainers, system administrators and developers. Hit me up on identi.ca or twitter if you know or a format or want to chat about what one would look like.
 digg it   seed it   del.icio.us   ma.gnolia
Comments: 0 Tags:        


Hello Guruplug You Jerk @ 2010-05-17 23:16:53.858363
Filed under: Linux  Tech  Frustration 
So you bought a Guruplug. You've waited and waited and now it's at your front door taunting you. You rip open the packaging and find a place in your networking closet to plug it in. Ethernet cord plugged in you plug the Guruplug in and watch your router logs. Nothing. Restart the plug and watch logs. Nothing. Change ethernet ports, reboot and watch logs. Nothing. Nothing. Nothing. ARRGGGG!

Don't fear -- here is some help. Before I start please note most of this information came from the forums at plugcomputer.org. Also note I am tired and cranky.

First, unplug the Guruplug and unplug the ethernet cord from the device. Before an ethernet connection will work you will need to ensure that your router/switch is 10/100 and not a gigabit router/switch. This may seem odd but many people have found plugging into gigabit devices (even those which can sense and fail downwards) causes the Guruplug to plainly not work. Go find an old router/switch and patch it in to your modern gigabit switch and then plug the guru plug in through there. While it shouldn't matter (at least I don't *think* it should matter) which ethernet jack you use note I have only tested it with the top one (the one closest to the logo and lights).

Lookie there! You see the Guruplug in your router logs now with an IP address. Great! ssh on over with the root user and the default password of nosoup4u (yeah -- really, that is it). Before you do anything else, STOP! Do you need the wifi device running? Anyone is able to join the network and use your connection! Even if you are going to be using the wifi in the Guruplug in your project you probably do not need it running at this point, so let's turn it off temporarily by running "uaputl sys_cfg_radio_ctl off". Use your desktop/laptop and check to make sure that you can no longer see the GuruPlug SSID (it may take a minute or two you to see). If you know you are not going to be using the Wifi for a while you can turn it off by editing /root/init_setup.sh and comment out everything from "rm -f /etc/wlanclient.mode" up to and including "/usr/bin/uaputl bss_start". The file should now look like this: 

# We always bootup in AP mode. Delete any stale files
#rm -f /etc/wlanclient.mode
#SSID=Plug2-uAP-`ifconfig eth0 | awk -F ":" '/HWaddr/ {print $6$7}'`
#
#insmod /root/uap8xxx.ko
#ifconfig uap0 192.168.1.1 up
#/usr/bin/uaputl sys_cfg_ssid $SSID
#/usr/bin/uaputl bss_start
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
iptables -t nat -A POSTROUTING -o ppp0 -j MASQUERADE
echo 1 > /proc/sys/net/ipv4/ip_forward
/etc/init.d/udhcpd start
/etc/init.d/dnsmasq start
iptables -A INPUT -i uap0 -p tcp -m tcp --dport 80 -j ACCEPT

# Re-enable bluetooth. In the earlier case, it didn't find the firmware.
#rmmod libertas_sdio libertas btmrvl_sdio btmrvl bluetooth 2>/dev/null
rmmod btmrvl_sdio btmrvl
/etc/init.d/bluetooth start

modprobe btmrvl_sdio
hciconfig hci0 up
hciconfig hci0 piscan
/usr/bin/mute-agent &

# Set leds
echo 1 > `eval ls /sys/class/leds/*plug*\:green\:health/brightness`
echo 1 > `eval ls /sys/class/leds/*plug*\:green\:wmode/brightness`
Now let's get that root password changed! We don't want someone iPhoning us! All you need to do is run "passwd". You will be prompted for your new password two times. Now the root password is no longer nosoup4u, it is what you just set it to. Don't forget it.

As any good engineer you just tried to update the system (you are a good engineer aren't you?). The problem here is that, well, it didn't work. 

Err http://10.82.108.51 binary/ Release.gpg    
  Could not connect to 10.82.108.51:80 (10.82.108.51). - connect (110 Connection timed out)
Err http://10.82.108.51 binary/ Translation-en_US
  Could not connect to 10.82.108.51:80 (10.82.108.51). - connect (110 Connection timed out)
Ign http://10.82.108.51 binary/ Release        
Ign http://10.82.108.51 binary/ Packages/DiffIndex
Ign http://10.82.108.51 binary/ Packages       
Err http://10.82.108.51 binary/ Packages       
  Could not connect to 10.82.108.51:80 (10.82.108.51). - connect (110 Connection timed out)
W: Failed to fetch http://10.82.108.51/kedars/sheevaplug_wifi/builds/packages/binary/Release.gpg  
Could not connect to 10.82.108.51:80 (10.82.108.51). - connect (110 Connection timed out)

W: Failed to fetch http://10.82.108.51/kedars/sheevaplug_wifi/builds/packages/binary/en_US.gz 
 Could not connect to 10.82.108.51:80 (10.82.108.51). - connect (110 Connection timed out)

W: Failed to fetch http://10.82.108.51/kedars/sheevaplug_wifi/builds/packages/binary/Packages  
Could not connect to 10.82.108.51:80 (10.82.108.51). - connect (110 Connection timed out)

E: Some index files failed to download, they have been ignored, or old ones used instead.
E: dpkg was interrupted, you must manually run 'dpkg --configure -a' to correct the problem.
As the error states, go ahead and run "dpkg --configure -a". Then edit /etc/apt/sources.list and comment out the silly internal to the manufacturers network IP address so the file looks like this: 

deb http://ftp.us.debian.org/debian/ lenny main contrib non-free
deb http://http.us.debian.org/debian stable main contrib non-free
deb http://security.debian.org lenny/updates main contrib non-free
deb http://www.backports.org/debian lenny-backports main contrib non-free
#deb http://10.82.108.51/kedars/sheevaplug_wifi/builds/packages/ binary/
Now run "apt-get update && apt-get upgrade" to get that system updated. This will take a bit of time. While that runs go learn about Mentally Ill Gangsters


OK, now we see all those stupid messages saying we are getting stuff from the future. That is cool and all but we don't need to hear about time traveling code over and over. Let's fix it using date. You will need to run date like so 'date -s "+Mon May 17 22:50:00 EST 2010"'. Now you will need to really set the timezone (assuming you are not in UTC). To do this replacing EST with your correct timzeone: "cd /etc/; mv localtime localtime.old; ln -s /usr/share/zoneinfo/EST localtime".

Now is the moment of truth. Reboot the plug by running "reboot". It will probably come back up with the same IP address (punch a tree in the face), but if you can't get to the plug check your routers logs to see if it got assigned another address. Everything should be great and you should be off to the races. YES DICE! Now I need sleep. I have no more caffeine powering my madness.
 digg it   seed it   del.icio.us   ma.gnolia
Comments: 0 Tags:      


Bad Java, BAD! No More Jars! @ 2010-03-27 23:26:42
Filed under: Code  Tech  Security  Frustration  Python 
A big frustration for me is the sprawl of Jar's (java "packages") which are everywhere. These special zip files tend to be copied into other applications and then left alone. Many of these Jar's have newer releases to fix security issues, but the bundled version isn't updated. It's even worse that many Jar's don't provide enough meta information that you can be sure of who owns it. Yes, you could keep SHA1SUM's in a database like maven does, and that is better than nothing, but it's not really a fix, it's a hack.

Here is an example of metadata that came with one Jar:

Manifest-Version: 1.0
Created-By: 1.6.0_10 (Sun Microsystems Inc.)

How helpful! Well, we can at least see what it is by the name of the file: sqlitejdbc.jar. It still doesn't tell us what version. Let's look at another:
Manifest-Version: 1.0
Archiver-Version: Plexus Archiver
Created-By: Apache Maven
Built-By: fberger
Build-Jdk: 1.6.0_06

Better, but still isn't that helpful. In this case we get lucky as some of the info is in the file name: gettext-commons-0.9.6.jar.

It really seems like the whole 'keep metadata in your Jar' is more of an inside joke which requires a hack to try to track what jars to include (as maven does). Welcome to Jar hell. How fun. Anyway, here is another hack ...

For the heck of it I decided to write a very simple scanner. It reads the metadata from the Jar file and then tries to match it up against an online database. If it gets no results back it keeps it as 'either safe or not enough information'. If there isn't even enough data to make a call out to the database it's assumed bad and tells you the user to bother about it if that is listed, and if the database confirms vulnerabilities it's known bad. It needs a lot of work to better guess information not provided by the metadata but here is an example run (with some changes to protect the guilty) 

$ python jarscanner.py *jar
WARNING:root:apache-mime4j-0.6.jar is the latest secure version or not enough info
WARNING:root:commons-codec-1.3.jar is the latest secure version or not enough info
WARNING:root:commons-logging-1.1.1.jar is the latest secure version or not enough info
WARNING:root:hsqldb.jar is the latest secure version or not enough info
WARNING:root:httpclient-4.0.jar is the latest secure version or not enough info
WARNING:root:httpcore-4.0.1.jar is the latest secure version or not enough info
WARNING:root:httpmime-4.0.jar is the latest secure version or not enough info
INFO:root:jetty-6.1.7.jar found 15 vulns
INFO:root:jetty-util-6.1.7.jar found 15 vulns
WARNING:root:servlet-api-2.5-6.1.7.jar is the latest secure version or not enough info
WARNING:root:The following jars are known to be bad ...
WARNING:root:jetty-6.1.7.jar
WARNING:root:jetty-util-6.1.7.jar
CRITICAL:root:Sorry, but a number of jars are crap and don't provide enough information.
    These should be assumed bad!!!
CRITICAL:root:bdiff.jar
CRITICAL:root:fast-md5.jar (go bug dragonlz about it)
CRITICAL:root:gettext-commons-0.9.6.jar (go bug fberger about it)
CRITICAL:root:jcip-annotations.jar
CRITICAL:root:linuxfolderwatcher.jar
CRITICAL:root:messages.jar
CRITICAL:root:snakeyaml-1.5.jar (go bug somov about it)
CRITICAL:root:sqlitejdbc.jar
CRITICAL:root:stringtree-json-2.0.9.jar
CRITICAL:root:swt.jar
CRITICAL:root:unixapi.jar
CRITICAL:root:XXXXXXXXXX.jar
CRITICAL:root:XXXXXXXXXX.jar


I'm tired. I'm going to bed. I'll throw the code up somewhere tomorrow.
 digg it   seed it   del.icio.us   ma.gnolia
Comments: 0 Tags:          


SonicWall Is Silly @ 2010-01-20 11:59:23
Filed under: Tech  Security  Frustration 
Ever hear of SonicWall? Sure you have! More than likely you have been a user in a coffee shop or at a restaurant (though you may have not known). The idea is to protect both the user and the local network from accessing content that is undesirable. Thankfully they tell you why the content has been blocked. Sadly, they block users from resources to help block scams. Why? Becuase they may not be smart enough to tell the difference. For instance, I had gone up to Phishtank to post up a phish email to warn others and SonicWall blocked me from accessing the site all the while allowing access to the VirusTotal which is a similar type of service but for file scans. You are probably saying "Ashcrow! It's obviously a mistake! Go, tell them to fix it!" and let me stop you there, because I did try. Two times actually and both times I received the same response:

Dear Customer:

You submitted the following rating request to SonicWALL CFS Support:
Rate phishtank.com as "27.Information Tech/Computers" at 2010-01-12 17:25:02.167

The request has been reviewed and rated as:
"28.Hacking/Proxy Avoidance" at 2010-01-20 03:35:03.190

You should see this rating change reflected within 1 to 3 business days.

Thank you for your request,
       SonicWALL CFS Support

At first I thought maybe they kept the same rating but opened access. Many people (and companies) don't know the difference between hacking, cracking, phishing and fraud. The next time I was out at the restaurant I tried hitting PhishTank. No dice.

So what does this mean? That is a hard one. It could mean a lot of things but the way I take it is that the folks making decisions on blocking content are nor proficient enough to tell the difference between a hostile site and one that helps protect users --- and that is NOT a good quality to have in a security content filtering vendor of any kind.
 digg it   seed it   del.icio.us   ma.gnolia
Comments: 0 Tags:      


RDU Airport Security Frustration @ 2009-11-25 05:19:40
Filed under: Personal  Frustration 
I usually will put up with a decent amount when it comes to travel. Lots of people are trying to get to many places and everyone is afraid they won't make it. I've been frustrated by the security checks at airports before, but today was a bit different. It started while I was waiting to put my stuff on the belt. Two security checkpoint guys were giving the instructions at the same time (though not saying the same thing at the same time) about 5 feet from each other. If I didn't know what to do (say, if I wasn't from the US) I wouldn't have been able to figure out what they were saying. So I get my bags on the belt and walk through the metal detector. Usually that means 'yay, you don't have anything crazy on you.' Not this time. I don't know if this was added in the last few months but I got a pat down to verify I didn't have anything. OK, so maybe it's an added safeguard, I'll go with that, but what happens next really irks me. My stuff 'clears' except for my backpack. I'm asked if I have a laptop in there. I explain that I don't, but there is a Nintendo Wii ... so he opens my bag and takes it out and states it has to be checked. The lady at the xray machine says 'pfff, I can't help what they put in their bags' and checks my Wii for bombs (... I assume). I stand and wait for my stuff to come through and it does after a few other bags. The lady behind the xray machine is pumping out reviews so much so that peoples stuff is getting pushed off the end of the belt. Luckily, I was able to get my stuff before it got to the end, but to do so I had to kick my shoes down the hallway while wearing my backpack, holding my laptop case and holding on to the items that I had to remove from my person/was removed from my bag.

I know there isn't a big screening process to work in airport security. I also know that I should put up with it because they get paid little and have frustrating jobs, but, on the other hand, I spent $550 on a plane ticket that last year at this time was about $200. Taking a train and being able to do work while I travel with less frustration (and, checking out Amtrak tickets online -- MUCH cheaper) seems to be looking better.
 digg it   seed it   del.icio.us   ma.gnolia
Comments: 0 Tags:    


 
A Django joint.
© 2007-2009 Steve 'Ashcrow' Milner | Studio7designs | Arbutus Photography