2012-07-03

Tkx in Perl

Putting this here so I don’t loose it.

To get Tkx to load without segfaulting, the following environment variable must be set.

export PERL_DL_NONLAZY=1

If it’s set, it works fine

Update: This can be set in a BEGIN block as follows:

BEGIN {
      $ENV{PERL_DL_NONLAZY} = 1;
}

Tcl Version of Blog generator

In my continuing re-implementing of things, I’ve also created a version of the site generation in Tcl.

Here’s the equivalent Tcl for generating the Table of Contents:

#!/usr/bin/env tclsh8.5

set output {}
foreach arg $argv {
    set fh [open $arg "r"]
    while {[gets $fh line] >= 0} {
        if {[regexp {^\#\#\#} $line]} {
            set date $arg
            regsub {^\#\#\# } $line "" line
            lappend output "<a href=\"index.html#$date\">$line</a><br>"
        }
    }
}

foreach line [lreverse $output] {
    puts $line
}

Simple Tcl Webserver

Here’s a simple webserver written in Tcl (Modified from here or here.

I added the ability to choose a port and a user and group to run as, when started as root.

# [Modified from] DustMotePlus - with a subset of CGI support
package require Tclx

set default   index.html

set port      [lindex $argv 0]
set user      [lindex $argv 1]
set group     [lindex $argv 2]
set root      "./public"

set encoding  iso8859-1
proc bgerror msg {puts stdout "bgerror: $msg\n$::errorInfo"}
proc answer {sock host2 port2} {
    fileevent $sock readable [list serve $sock]
}
proc serve sock {
    fconfigure $sock -blocking 0
    gets $sock line
    if {[fblocked $sock]} {
        return
    }
    fileevent $sock readable ""
    set tail /
    regexp {(/[^ ?]*)(\?[^ ]*)?} $line -> tail args
    if {[string match */ $tail]} {
        append tail $::default
    }
    set name [string map {%20 " "} $::root$tail]
    if {[file readable $name]} {
        puts $sock "HTTP/1.0 200 OK"
        if {[file extension $name] eq ".tcl"} {
            set ::env(QUERY_STRING) [string range $args 1 end]
            set name [list |tclsh $name]
        } else {
            puts $sock "Content-Type: text/html;charset=$::encoding\n"
        }
        set inchan [open $name]
        fconfigure $inchan -translation binary
        fconfigure $sock   -translation binary
        fcopy $inchan $sock -command [list done $inchan $sock]
    } else {
#        puts $sock "HTTP/1.0 404 Not found\n"
        puts $sock "Error - Path was: $name"
        close $sock
    }
}
proc done {file sock bytes {msg {}}} {
    close $file
    close $sock
}
socket -server answer $port
id group $group
id user $user
puts "User/Group changed to [id user] : [id group]"
puts "Server ready..."
vwait forever

I’ll probably try serving this website for a little while with this.


Fedora vs Ubuntu

As I’ve been playing around with Tcl, I’ve been struck with how much more up to date the default Tcl and libraries seem to be in Fedora rather than Ubuntu.

The default tcllib and Tclx packages in Ubuntu are still based on Tcl 8.4.

I think that’s probably at least 5 years old.

Standard

2012-06-30

Google Chrome on Fedora

Given my recent frustration with Firefox, I wanted to install another web browser.

It doesn’t appear that either Google Chrome or chromium are available in the standard Fedora repository. You can add a repository to get Chromium or Chrome.

However, it turns out that Google chrome will download directly from the Chrome website as an rpm package that you can install directly. That was nice and easy.

I don’t know how that effects tools like yum. Maybe I’ll see if it looks like yum is now aware of Chrome.


Perl unshift Operator

I had forgotten about Perl’s unshift operator. It allows you to put something at the beginning of an array (instead of at the end with push).

As a result, using unshift instead of push, I was able to get rid of the reverse operation that I needed to get the table of contents into the proper order before printing it out.

Standard

2012-06-29

Perl Web Programming

It turns out it’s actually pretty easy to do web programming in Perl using PSGI once I waded through the documentation. The documentation makes things seem complex, but it’s actually pretty easy once you get down to the essentials of what you get as a request and what you need to return.

Maybe I’ll try to write up the tutorial I would have wanted to read.


Firefox Frustration

I’ve been trying out Firefox again since it seems to be the main browser by default in the Fedora repositories. However, I keep getting odd hangs while using it. I don’t know what’s inducing it – it seems to happen when I’m trying to drag tabs around.

So I guess, I’m just going have to learn how to add the repositories to get chromium, or see if I can download Google Chrome directly.


Static Web Site App in Perl

Here is the little app I wrote in Perl using PSGI (specification) to serve my website.

Currently I am running it using Starman but that may change if I get Apache up and configured for virtual hosts.

my $app = sub {

    $doc_root = "./public";
    my $env = shift;
    $path = $env->{PATH_INFO};
    $file = $path;
    $error = 0;

    $file =~ s/\.\.//g;

    if ($file =~ m(^/$)) {
        $file = "index.html";
    }

    if (-f "$doc_root/$file") {
        open FILE, "$doc_root/$file";
    } else {
        $error = 1;
    }

    if (! $error) {
        return [
            '200',
            [ 'Content-Type' => 'text/html' ],
            \*FILE,
                ]
        ;
    } else {
        return [
                '200',
                [ 'Content-Type' => 'text/plain' ],
                [ "Error - Path was; $file" ],
                ]
        ;
    }
}

Perl Web App Update

So with the above code, I no longer have to map to the exact files that I was wanting to serve. I think I’ve got security set so that only the document root and below is accessible. Howerver, I"m kind of a newbie at this, so I might have left a huge gaping hole for hackers to enjoy.


PSGI / WSGI / Rack

Having spent the day playing around with PSGI, I became curious about the alternatives (and inspirations) in the Ruby and Python space. It’s pleasing to discover that their concepts are basically the same as the Perl implementation. That’s cool, because it means I could begin programming web apps according to those specifications without having to learn a whole new framework of thinking.

That means that if I want to play around with web programming in Ruby, I can reuse most of this learning that I’m doing so far in Perl. Which is nice, because while I haven’t been a big fan of Rails, I do like Ruby, and I like PSGI, so from a brief glance at Rake, it looks like I would like it as well.

If it came down to doing web stuff in Ruby, maybe I could work on the Rake layer of web stuff, and my friends could work in Rails.


Perl Table Of Contents Script

I reworked the table of contents script from the other day to be a little less ugly. Now instead of stringing together Perl commands on the command line, the shell script calls this simple Perl script:

#!/usr/bin/env perl

@output = [];
while (<>) {
    if (/^###/) {
        $date = $ARGV;
        chomp;
        s/### //;
        push @output, qq(<a href="index.html#$date">$_</a><br>\n);
    }
}

print foreach (reverse @output);

I think this is a lot cleaner than what I had previously:

perl -lne '
  /^###/ && print "$ARGV:$_";
' * | perl -e '
@ls = <STDIN>; print foreach (reverse @ls);
' | perl -lpe '
s/### //
' | perl -lna -F: -e'
print qq(<a href="index.html#$F[0]">$F[1]</a><br>)
' > ../toc.middle

Anyway, so far I’m happy with the change.


Emacs Frustration

I’m getting annoyed with the way Emacs is mixing tabs and spaces in my writing and code. I’m going to have to go look up how to keep it from doing that again. That and fixing the broken backspace / Ctrl-H behavior are two things that I have to re-figure out each time I really start using Emacs in a particular environment.

In particular, this is a pain, because when I try to indent code so that Markdown will treat it specially, Emacs makes it all kinds of weird with mixed spaces and tabs, which then confuses Markdown as to where the lines should actually be.

So for example, the above (first) Perl snippet, the if statement should be indented (and it is visually in the Markdown source). But because it’s indented with tabs, where the surrounding lines are indented with spaces (Emacs doing, not mine), it doesn’t translate the indentation correctly to the code block.

Standard

2012-06-28

Possible Blog Downtime For Server Migration

I’m getting ready to change the server that serves this blog from Arch Linux to Fedora Linux. At some point in the next day or so that will probably mean this blog is down for a bit. Of course, since it will be down, you probably won’t be able to read this notice. Luckily, the audience for this blog is probably in the 1-3 person range.


Fedora is Alive

The server is now running on Fedora 17.

It will be interesting to see whether I end up likeing an rpm based distro or not. I’ve almost always used apt based distros in the past for personal stuff except for forays into Arch and Slackware. But so far I like the experience.

It’s possible that all package managers have gotten to the point where there’s not really any issue with using any of them.


Simple Web Servers

With the new server online, I wanted to find a quick webserver that could just serve static pages out of a directory to get the blog back up. Since I had rewritten the blog generation in Perl, I kind of wanted to find something written in Perl to serve the content.

It turned out to be more difficult to get exactly what I wanted than I was hoping.

That may be because I had several requirements that I hadn’t made explicit.

  • I wanted a webserver that could run on port 80, but could drop priviledges to a normal user after being started.
  • I wanted it to be able to serve from an arbitrary directory.
  • I wanted it to be essentially a program or a runner for a program that I would write.

Basically I wanted something equivalent to the Node.js http-server that I had modified. Python’s SimpleHTTPServer came the closest to what I wanted initially, but I couldn’t figure out how to get it to meet my desire to have the process drop priviledges when started as root.

I would have thought that I could find a simple web server written in Perl that would do what I wanted. And I did eventually accomplish my goal. However, none of the Perl web server frameworks were quite as simple by default as I was hoping for.

The closest to what I wanted, and what I ended up using, is Starman. However, I still had to write a little PSGI app, and in so doing, couldn’t figure out how to get a default index page without having to map all pages and a default to their respective files.

There were two packages that came so close to doing what I wanted:

  • Plack::App:File
  • Plack::App::Directory

However, neither would show the default index.html when it wasn’t explicitly specified. Plack::App:File just returned a "file not found" type error and Plack::App::Directory returned a directory listing. Basically what I wanted, and what I might see if I could modify, is Plack::App::Directory, but with the default action to return an index.html if it exists instead of the directory listing.

Standard

2012-06-26

Ruby

Since I’ve been playing with re-implementing things, I went ahead and also implemented the site rendering script in Ruby.

I also got a fair way to re-implementing the table of contents generation in Ruby, but I’m having a bit of trouble with getting the current working filename to output.

So far my thoughts are …

Ruby looks a little cleaner than Perl to my eyes for most statements. However, there are places where I couldn’t figure out how to do things as concisely as I can in Perl. And although Ruby looks cleaner, I don’t know that it’s actually clearer than the Perl code. Where in Perl, the language itself gives you a fair amount of clues about what operations are actually going on, in Ruby you really need to know what class / object you are talking about to understand what the methods are doing.

Sometimes, the arguments to a Ruby object method really don’t make sense out of the context of knowing that the method takes something as a special case to change it’s behaviour. For example, gets takes nil as an argument to mean that it should read all lines available from the file or input stream.

One could argue that Perl requires you to recognize what context (list or scalar) you are in to understand the behaviour of the operator, but in my experience that is usually fairly evident from the immediate surrounding code (although to be fair, that may also require looking up function definitions that you are not familiar with).

Standard

2012-06-25

TOC Perl Reimplementation

This morning I reimplemented the code that generates the archive listing in Perl.

So the following bash script:

#!/bin/bash
cd entries
grep '###' * | tac | sed -e 's/### //' | awk -F: '
{print "<a href="index.html#">" $2 "</a><br>"; }
' &gt; ../toc.middle

became the following using Perl instead of the specific bash commands:

#!/bin/bash
cd entries

perl -lne '
/###/ &amp;&amp; print "$ARGV:$_";
' * | perl -e '
@ls = ; print foreach (reverse @ls);
' | perl -lpe '
s/### //
' | perl -lna -F: -e '
print qq(<a href="index.html#$F[0]">$F[1]</a><br>)
' &gt; ../toc.middle

So, the Perl, is longer, and not necessarily any faster, because I’m actually executing a seperate instance of Perl for each of the bash commands I’m replacing. So, it could conceivably be even slower than the original bash script.

But it was interesting to do.

BUGS

The previous post revealed a bit of a bug with the archive listing generation. The fact that the scripts in the post had the symbols that were being searched for, meant some false entry listings were generated using those lines from the post, instead of just the post titles.

Also, I noticed that the version of Markdown that I’m using with Perl (the original John Gruber version) does not support a syntax that I was using in the Node.js version to indicate code blocks.

So, I need to look back through previous posts for any that used that syntax and change them, otherwise they won’t display correctly with the new site generation.

Update: Archive BUG fixed

I fixed the bug in the archive listing.

I still need to look through all entries for the previous version of code block syntax.

Update 2: Code Blocks fixed

Went through and changed all previous entries with code blocks to use the original markdown code block syntax.

Good and Evil in Perl

I had a thought today about Ruby, PHP and Perl while reading a post on programming languages. It reminded me of the movie Twins…

  • It seems like Ruby took as much of the good stuff from Perl as it could.
  • It seems like PHP may have ended up with the bad stuff.

But Perl gives you both good and evil in one complete language 🙂

Perl vs Ruby = Functional vs Object-Oriented

Having just recoded this blog in Perl, I’ve been wondering also how the code would look in Ruby.

One thing I’ve been thinking about today regarding Perl and Ruby, is that once you go beyond a simple procedural paradigm, they differ significantly in what emphasis they make easy.

So, for example, I know you can do object-orientation in Perl, but I personally haven’t wanted to deal with it for my programs, whereas in Ruby, object-orientation is a reasonable default.

However, Ruby is <em>so</em> object oriented that while it provides some constructs that can mimic functional programming, you are really still dealing with an Object paradigm, and I personally don’t know how to do really functional type things, whereas Perl really makes it quite easy to program in a functional style, with good support from the language.

So basically Ruby tends toward being an Object-Oriented scripting language, and Perl tends toward being a Functional scripting language, once you move beyond simple procedural scripting.

I like some of the look of Ruby better than Perl, but I’ve tended to think in a more Functional than Object-Oriented mindset for the last several years, so I actually may prefer Perl for what it can do in a functional style.

Interesting Perl Discussion

Some interesting discussions:

Is Perl still a useful viable language?

Standard

2012-06-24

Perl Re-implementation

I’ve just finished re-implementing the blog rendering engine in Perl. It’s partly an experiment, and partly because I’m interested in Perl these days as kind of a workhorse language.

The Perl site generation actually takes longer than the Node.js version did when running, but I’m pretty sure that’s because for the moment I’m just spawning a separate process for each entry to convert the entry’s markdown to HTML. I believe that was all in process in the Node.js version.

I will probably try to move that into the main Perl process when I get tired of the 1 or 2 second running time that the Perl process takes.

Perl Table of Contents

I also want to try re-implementing the bash script that I use to generate the archive listing in Perl. I’m curious how much more verbose it will be in Perl. Interestingly, Perl is the only language I’ve even been willing to consider re-implementing it with, since every other language I’ve played with, would have been so much more verbose for the same functionality that I just couldn’t bring myself to attempt it.

Standard

2012-06-22

Java and Perl

If you just wanted a toolkit that was focused on "getting things done", I think it would be hard to find a better one than the combination of Java and Perl. Between the two of them, I think most (at least business oriented) problems could be addressed between them. Both Java and Perl have a huge amount of working already existing code to address almost any issue you could encounter.

Neither of these languages are really "cool" languages any more, and I have previously had a bit of disdain for Java. However, between the two of them, I think one could make a pretty decent stab at an appropriate (and fairly quick) application for almost any normal problem area.

I would probably ultimately supplement them with C for low-level stuff. And I might pull in something like Tcl/Tk for really quick and dirty gui tools, but the core of what one needed to accomplish could be done it most cases by one or the other of Perl or Java.


Further thoughts

Thinking about Perl as a primary scripting language for problem solving may only make sense if one has already come from a Unix background, where Perl basically encapsulates the experience one probably already has from Awk, Sed and Bash.

Without that background, there may be other choices that make more sense. Python is arguably a simpler and cleaner(?) language, and probably easier to learn for someone that doesn’t have a previous Unix background.

However, I don’t think there’s any other scripting language than Perl, that has the level of support for easily integrating already existing code for a problem domain and quickly deploying it.


C++ and Python

It seems to me that the application domains that I would posit Java and Perl for could also mostly be addressed by C++ and Python.

I think that Perl still has an edge on Python for stuff like parsing text files. But Python and C++ may have an edge on creating decent application GUIs over Java or Perl. In addition, C++ is trivially able to handle the low-level problems that require bringing in C in addition to Java and Perl.


PrayPal

I think an interesting name for a service would be PrayPal, some kind of online prayer request thing where you could submit a prayer request and know (trust) that someone was praying for it.


ImageMagick utilities

I’ve been looking at ImageMagick over the past couple days.

Here are the utilities that so far have been useful to me:

  • import – I’ve used this to take screenshots. It’s unfortunate that this doesn’t work without an X server on Windows.
  • convert – used to actually manipulate images: resize, change formats, etc.
  • identify – show information about an image.

I’m having to use identify to obtain the image geometry for a screenshot so that I can calculate the correct offset to crop a small portion of the image for OCR.

Standard

2012-06-21

Java Native Access

Java Native Access (JNA) is an alternative to Java Native Interface (JNI) that does not require writing C/C++ wrapper code – it basically allows a ffi style calling ability for C functions contained in dlls.

I had long had the view that it was unfortunate that it was so hard to directly interface Java with lower level code. This seems like for simple cases it makes this much simpler, and for hard cases, there is JNI.

Standard