pyfunc

Python has a solid backbone of modules for various operations. It would be a pity if we had to write one-liner python programs just because we cannot avail those functionalities on the command line - wouldn’t it?

Day 1:

I wanted to query a heavily nested json for a specific key. Now jq does the job pretty well, but the easier way was to flatten the json and just grep out the required key. I had a flatten json python function already written. So all I did was to write a simple program to use that function and flatten the file passed as a command line argument. Good enough.

Day 2:

I wanted to find out if one file is a subset of the other. I looked it up and saw that this post describes a way to do so:

$ cat > a
1
2

$ cat > b
1
2
3

$ cat b | sort -u | wc
3 3 6

$ cat a b | sort -u | wc
3 3 6

as both of the above results are same - a is a subset of b. Pretty neat but then I would need to compare the two results and write a shell script todo so. The python library provides the set data structure having the very intuitive issubset function which just does this. So all I did was to write a simple program to use that function and do the necessary check. Does the trick.

Day 3:

A colleague was mangling some text and asked me how to convert uppercase to lowercase. I showed him how:

$ echo TEST | tr '[A-Z]' '[a-z]'
test

He asked is there a similar functionality to swap case? I did not have a ready answer to do so on the command line but told him that he could write a one liner python script using string.swapcase to do so. So all I did…

Now wait a minute…

Why would I want to write separate programs to flatten a json, check if a file is a subset of the other or swap case when I could very written write one program which takes the target module, function to call, its arguments and does the right thing?

Enter pyfunc

Now all of the above problems are solved by doing:

$ cat /var/tmp/meal.json
{
  "sandwich": {
    "ingredients": {
      "breads": {
        "fancy_name": {
          "price": "exhorbiant",
          "taste": "exotic/oaky with a savory aftertaste"
        },
        "just_bread": {
          "price": "affordable",
          "taste": "like bread"
        }
      }
    }
  }
}

$ pyfunc -m pyfunc.utils.json_flatten -a /var/tmp/meal.json  | grep 'fancy_name.price'
sandwich.ingredients.breads.fancy_name.price: exhorbiant

$ echo TeSt | pyfunc -m string.swapcase
tEsT

$ pyfunc -m pyfunc.utils.f_is_subset -a /var/tmp/f1 /var/tmp/f2
True

I get everything in the standard library as a bonus:

$ pyfunc -m range 1 7 2
1
3
5

For the commonly used pyfunc calls I convert them to aliases:

$ alias jsonflatten='pyfunc -m pyfunc.utils.json_flatten -a'
$ alias fsubset='pyfunc -m pyfunc.utils.f_is_subset -a'

And now I have a richer command line:

$ cat /var/tmp/meal.json | jsonflatten | grep 'fancy_name.price' | cut -f2 -d':'
exhorbiant

More examples in the github repo - pyfunc

The only glitch that I faced while writing this was to accomodate different types of arguments being passed to the target functions like list, dict, etc. I added support for method signatures to do so, but now I feel it would be easier to have pyfunc functions to just take args and pass in a json from the command line. I will be adding that soon.

pip pyfunc up and bring the awesomeness of python on the command line!