DEV Community

Barrett Otte
Barrett Otte

Posted on • Edited on

Simple Util to Pull Code From IBM i

#python #ibmi #rpgle #as400

Alt Text

In one of my really dumb side projects I'm making, I have some IBM i code that I'd like to keep in my git repository with some other stuff. I could use the Integrated File System (IFS), but truthfully I don't know enough about IBM i yet to use it correctly.

So, in the meantime I have a quick and dirty python script that lets me pull multiple source members based on a config file, repo.json. This isn't the cleanest, but it works pretty well.

My repo.json is very basic, but lays out a basic IBM i file structure to loop over below with FTP

{ "library": "BOLIB", "spfs": [ { "name": "QRPGLESRC", "extension": "RPGLE", "members": [ { "name": "FIZZBUZZ" } ] } ], "output": "./" }
Enter fullscreen mode Exit fullscreen mode

Basic Idea

If you connect to IBM i over SSH you can navigate and find your source members. For example, I'm going to drill down into my QRPGLESRC file within my BOLIB library.

Alt Text

As you can see, there are my QRPGLESRC members hanging out. All my script does is automate grabbing the source members.

The Code

To start, I use a few basic modules found in the python standard library.

import ftplib # easily setup FTP connection import json # read in repo.json config file import getpass # used to discretely get password import os # used to safely make directories for output
Enter fullscreen mode Exit fullscreen mode

I load my config file, instantiate my FTP client, and get the hostname, username, and password from the console

config = {} with open("./repo.json", 'r') as f: config = json.load(f) ftp_client = ftplib.FTP() host = input("Enter Host: ") user = input("Enter User: ") password = getpass.getpass("Enter Password: ") # FTP logic below...
Enter fullscreen mode Exit fullscreen mode

Next, I setup my FTP connection shell with very basic error handling

try: ftp_client.connect(host, timeout=10000) ftp_client.login(user, password) # The meat of the code here ... except ftplib.all_errors as e: print("Error occurred with FTP.\n" + str(e)) exit(1) except Exception as e: print("Some other error occurred\n" + str(e)) exit(1) finally: ftp_client.quit()
Enter fullscreen mode Exit fullscreen mode

Now the fun stuff, sending FTP RETR command to get data from IBM i

lib = config['library'] # my example only has one library, so no looping  for spf in config['spfs']: print("Fetching member(s) from {}/{}".format(lib, spf['name'])) if not os.path.exists('./'+spf['name']): os.makedirs(spf['name']) # make a directory based on source physical file name  for mbr in spf['members']: resp = [] # The magic command to get data ex: BOLIB/QRPGLESRC/FIZZBUZZ.mbr  cmd = "RETR {}".format("/QSYS.lib/{}.lib/{}.file/{}.mbr").format( lib, spf['name'], mbr['name']) ftp_client.retrlines(cmd, resp.append) # run the command  # Create file based on specified extension ex: RPGLE  filepath = spf['name'] + '/' + mbr['name'] + '.' + spf['extension'] # Finally, write data to file  with open(filepath, 'w+') as f: for line in resp: f.write(str(line) + '\n') print(" Saved " + filepath)
Enter fullscreen mode Exit fullscreen mode

Final Script 

import ftplib # easily setup FTP connection import json # read in repo.json config file import getpass # used to discretely get password import os # used to safely make directories for output  config = {} with open("./repo.json", 'r') as f: config = json.load(f) ftp_client = ftplib.FTP() host = input("Enter Host: ") user = input("Enter User: ") password = getpass.getpass("Enter Password: ") try: ftp_client.connect(host, timeout=10000) ftp_client.login(user, password) lib = config['library'] # my example only has one library, so no looping  for spf in config['spfs']: print("Fetching member(s) from {}/{}".format(lib, spf['name'])) if not os.path.exists('./'+spf['name']): os.makedirs(spf['name']) # make a directory based on source physical file name  for mbr in spf['members']: resp = [] # The magic command to get data ex: BOLIB/QRPGLESRC/FIZZBUZZ.mbr  cmd = "RETR {}".format("/QSYS.lib/{}.lib/{}.file/{}.mbr").format( lib, spf['name'], mbr['name']) ftp_client.retrlines(cmd, resp.append) # run the command  # Create file based on specified extension ex: RPGLE  filepath = spf['name'] + '/' + mbr['name'] + '.' + spf['extension'] # Finally, write data to file  with open(filepath, 'w+') as f: for line in resp: f.write(str(line) + '\n') print(" Saved " + filepath) except ftplib.all_errors as e: print("Error occurred with FTP.\n" + str(e)) exit(1) except Exception as e: print("Some other error occurred\n" + str(e)) exit(1) finally: ftp_client.quit()
Enter fullscreen mode Exit fullscreen mode

The File was Fetched!

This is stored in ./QRPGLESRC/FIZZBUZZ.RPGLE

 /free // The classic fizzbuzz program in RPGLE Free dcl-s num int(10); for num = 1 to 100; if %REM(num:3) = 0 and %REM(num:5) = 0; dsply ('num - ' + %char(num) + ' FIZZBUZZ'); elseif %rem(num:3) = 0; dsply ('num - ' + %char(num) + ' FIZZ'); elseif %rem(num:5) = 0; dsply ('num - ' + %char(num) + ' BUZZ'); else; dsply ('num - ' + %char(num)); endif; endfor; *INLR = *ON;
Enter fullscreen mode Exit fullscreen mode

Simple Batch script

In my repository, I made a little batch script so I could pull IBM i code and commit it with the rest of my repository in one call.

@ECHO OFF IF [%1] == [] GOTO NOMSG python ibmi-pull.py && git add . && git commit -m "%~1" && git push origin master GOTO END :NOMSG ECHO "Enter the commit message!" :END PAUSE
Enter fullscreen mode Exit fullscreen mode

Again, there's probably some better "tools" you could make involving the IFS, but I'm just not there yet knowledge-wise.

As an experiment, I expanded upon this simple script to grab an entire library and generate a basic git repository :
https://github.com/barrettotte/IBMi-Lib-Repo

Thanks for reading.

Top comments (2)

Collapse
 
gajendertyagi profile image
Gajender Tyagi •

Nice! I absolutely loved what you did there.
I tried similar thing just with nodejs. What i was doing instead of a repo.json, I fetched all the source-pf and members name from the library (response was list in json format). Then i traversed through the json to fetch source, that way i dont have to write myself what to fetch.
One issue which I faced was with conversion of data from EBCDIC to ASCII, which i was not able to do in ssh fetch command.
What you have used is an IBMi FTP command RETR so i guess it doesn't need any conversion paramters.

Collapse
 
barrettotte profile image
Barrett Otte •

Thanks! One day I hope to get around to finishing rewriting my new version of this util.

Unfortunately, this also has that darn EBCDIC to ASCII conversion problem. My new util uses an SFTP module, which allows you to specify encoding. In my case I started using IBM037

Thanks for commenting, good to see some other IBMi folks on here!