DataPower Commander v0.3 – more useful than ever

DataPower Commander v0.3 – more useful than ever

DataPower Commander (dpcmder) is a command line tool I’ve created for easier maintenance of files on DataPower appliances and easier development of DataPower appliance solutions. You can use dpcmder too – it is an open source project available on GitHub (croz-ltd/dpcmder).

It is developed using go (golang) which makes cross-platform releases much easier and cross-compilation is built in as a go tool feature

Since last time I wrote about dpcmder so many new features were added that I felt compelled to write a new blog and describe how dpcmder can make your day nicer and your work with DataPower easier. Let’s be honest, it will make your whole life easier

The most interesting new features are:

• DataPower appliance configuration list
• domain export
• object maintenance mode
  • view object configuration
  • edit object configuration
  • delete object configuration
  • view non-saved changes in the object configuration
  • save current domain changes
• nicer input/progress dialogs
• status messages history

A bit more comprehensive list of new features will be given below but let’s help you start using dpcmder before we come to that.

dpcmder new input dialogs

Getting started with dpcmder

DataPower Commander works under Linux, macOS and Windows (normal Windows cmd prompt) but since there are no default file viewer/editor/diff externals command-line programs available under Windows it is a bit “harder” to do the initial setup for Windows environment. I would suggest you to use WSL or cmder environment to make all features of dpcmder easier to use.

The best way to start using dpcmder is to run it with the “-h” flag and see all other flags available. After that just call with appropriate flags to connect to your DataPower appliance, for example:
dpcmder -s https://127.0.0.1:5550 -u admin -p admin -c LocalDp

This command will connect to your local DataPower instance which exposes its SOMA interface at port 5550 using the default username and password combination and will save this configuration under given configuration name “LocalDp”. Don’t worry if you forgot to add “-c” flag – just opened DataPower configuration is saved as “_PreviousAppliance_” in the DataPower appliance list where you can easily clone it to whatever configuration name you need using F9/9 key.

You can even start without using any command line parameters when starting dpcmder for the first time and create a new DataPower configuration using the F8/8 key (when the [empty] list of appliances is shown). However, this option is not recommended if you plan to save your password to dpcmder configuration file (~/.dpcmder/config.json) – passwords are saved in the configuration file as base32 encoded text so you would have to properly encode password value before saving it (echo -n "myPassword" | base32).

If you need more information you can pass flag “-help” to dpcmder and it will write full dpcmder help to console so you can get more information about its features and their use.

Note on appliances configuration password safety

If passwords are saved in a dpcmder configuration file (if they are passed to dpcmder using “-p” flag) they will be base32 encoded so anyone with access to your dpcmder configuration file would be able to decode it easily. If you cannot guarantee the safety of this file, please avoid using the “-p” flag and enter a password when you connect to each appliance – this password will not be saved into the dpcmder configuration file.

DataPower appliance list

When you open DataPower commander without any parameters a list of saved DataPower appliance configuration is shown (which can be empty if no appliances were added through command-line flags or by using F8/8 key).

You can connect to DataPower using a given configuration (Enter key), view current configuration (F3/3 key), edit the current configuration (F4/4 key), clone current configuration (F9/9), create a new empty configuration (F8/8 key) or delete current configuration (Del/x key).

If the configuration for DataPower appliance you want to access doesn’t contain password information, the first time you want to connect to that appliance (in running dpcmder program) a dialog for password input will be shown.

If you run dpcmder using proper SOMA or REST connection parameters on command line DataPower appliance list will not be shown – list of domains on DataPower you wanted to connect to (using command line parameters) will be shown instead. You can easily go back to the DataPower appliance list by going to the parent view (selecting “..” item).

List of DataPower appliances

DataPower domain export

One of the new features  which I felt to fit most naturally into dpcmder is exporting the DataPower domains. Just as it is possible to copy files and/or directories from DataPower to the local filesystem it feels natural to copy the whole domain to a local filesystem. The same key is used for that purpose (F5/5 key) and the result of that operation is the DataPower exports zip archive created on a local filesystem.

The exported archive contains all objects and files from the selected DataPower domain.

I am considering to add function to import domain (and even delete domain) but since these operations are quite powerful there is a bigger chance to make some “damage” to your appliance using those. However, these options are something that would nicely round up features of working with DataPower domains so these will probably be added in some future version of the dpcmder.

Exporting DataPower domain

DataPower object maintenance mode

Object maintenance mode is very useful for making small changes to existing DataPower object configuration or viewing configuration changes (SOMA only). For example, changing the port number used for XML Firewall Service. Or maybe you don’t want to change anything but just want to quickly check the current value of some configuration option.

In the new release of the dpcmder, 0 key is used to switch between (default) “file” mode and “object” mode. To use object mode we have to connect to the DataPower appliance and select one of the domains available.

When we switch to object mode a list of all “class” of objects for which at least one object in the selected DataPower domain exists is shown. Instead of file size count of available objects is shown. In case we used the SOMA management interface to access DataPower classes we mark all modified classes with “*” (if at least one object is new, modified or deleted for that class).

The same key (Enter key) used for the operation of the “entering” domain/filestore/directory is used to show all available objects of the selected object class. In case we used the SOMA management interface we show if an object is new, deleted or modified (and can even view changes for a modified object using “d” key). That operation shows us names of all objects of the selected class and enables us to do more advanced operations on each of those objects:

• view object (JSON/XML)
• edit object (JSON/XML)
• delete object
• view unsaved object changes (XML)

In case we used the REST management interface to connect to the DataPower appliance object will be accessible using JSON, in case we used SOMA management interface XML will be used. The best feature we get “for free” is that if our editor (or viewer) supports syntax highlighting for JSON/XML we will have nicely highlighted code making our viewing and/or editing of DataPower configuration easier.

DataPower list of object classes used

DataPower list of objects of a selected class

Edit DataPower object (REST / JSON example) using vim editor

View DataPower object unsaved changes (SOMA-only) using external diff command

List of interesting changes since the last blog

For a more comprehensive list of changes you can check releases history but here is a summary of changes:

• fixes
• DataPower appliance configuration saving and loading
• diff command (comparing files/directories)
• better error handling in calls to external programs (+ hints how to fix those)
• better in the program help
• shows program help in console (“-help” flag)
• creation of a new (empty) file
• view status history (“m” key)
• replaced library for terminal interaction (termbox-go -> tcell) for better cross platform support
• added nicer terminal dialog and error messages
• blinking status when sync mode is on
• added object mode (edit and delete operations available)
• editing DataPower files can use syntax highlighting available in the external editor
• workaround enabling to use of diff command (normal, non-blocking) – custom blocking command is still recommended

Features of dpcmder

• appliance configurations
  • saves and loads DataPower appliance configuration to enable fast connections to appliances
  • edit/clone/create/delete DataPower appliance configuration
• file maintenance
  • view, edit, copy and delete file hierarchies (DataPower and local file system)
  • filter and search files in the current directory
  • comparing files/directories (DataPower vs. local file system)
• objects maintenance
  • switching between filestore mode and object mode (0 key)
  • existing objects can be quickly edited (as JSON (REST mgmt) or XML (SOMA mgmt) using current editor)
  • existing objects can be quickly deleted
• domain export
  • when applying copy operation to DataPower domain it is automatically exported as zip files (exporting all local files)
• sync mode
  • turn on to automatically upload new and changed files from the local filesystem to DataPower
  • useful for development to automatically propagate your changes from any IDE/editor you are using to DataPower

REST management vs. SOMA management

Some of the later features added were available only if we use the SOMA management interface to connect to the DataPower appliance. While the REST management interface seems a bit nicer some features are not available and/or documented. For example, Diff operation seems to be available through REST but I wasn’t able to find any documentation about that action or find a way to use it. From 154 actions available through /mgmt/actionqueue/{domain} (IDG.2018.4.1.3) only 3 are documented in IBM’s Knowledge Center:

• LoadConfiguration
• Export
• Import

Some functions of DataPower object mode work a bit differently in REST vs. SOMA mode – for example in the latest version available on the Git master branch, if the object’s admin state is down we can’t make changes to it using a REST management interface (REST call returns the HTTP error 409 Conflict).

Issues mentioned above are reasons why I would suggest you to use the SOMA management interface for your DataPower management – REST is nice but it still doesn’t offer everything SOMA does.

So Long, and Thanks for All the Fish

There are still many new features we consider adding to dpcmder so its development will continue further. For example – currently, dpcmder has a feature to edit or delete DataPower objects but doesn’t have a feature to create a new object (it would be a bit hard to use). The next version (already on Git master) will be able to clone the existing one or even copy the object configuration to a local file or create/update the DataPower object from the local configuration file. Currently, dpcmder can create a new domain but can’t delete the existing one, it can export the DataPower domain but can’t import it – these are some of the areas we consider for dpcmder further development.

The last version seems to be much nicer and contains a lot more functions compared to the first release but this doesn’t mean DataPower Commander is complete so stay tuned, there will be new features for dpcmder in 2020.