About otc-tools (deprecated)

This is the otc tool for using the Open Telekom Cloud APIs. It is a simple demonstrator using shell/curl/jq to use the OpenTelekomCloud's OpenStack and OTC specific REST interfaces. It is used for testing by the OTC engineering team and can be used as a blueprint for own tooling. Unlike the more extensive tools, this one does not pull in a load of dependencies.

Prefix any command by --debug to observe the REST calls being made.

otc-tools is NOT recommended for production use. We do fix bugs that get reported to us and do accept contributions (pull requests / patches) if they meet our standards, but we do not actively develop this tool any further except for internal needs.

For production use, we recommend using the native OpenStack python client tools (python-openstackclient) with the plugins from python-otcextensions that implement OTC specific functions. This is also what we preinstall on our latest public Linux images. It's also still possible to use the older service specific client tools (such as python-novaclient etc.) and complement this by python-otcclient or otc-tools, but that is deprecated now. All of the OTC-specific tools can be found on github/OpenTelekomCloud.

Of course higher level tools, such as heat, ansible or terraform are a good choice for automation. For a more programmatic approach, use openstacksdk (and shade).

The otc-tools repository (and package) also has two wrappers for openSSH, ssh_otc.sh and scp_otc.sh that allow you to specify source/targets by VM name or VM UUID instead of IP address. It uses otc.sh to resolve name/ID to IP address behind the scenes (and has some cleverness to know which IP to use in case of several IPs and to chose an EIP if needed).

Usage

Set your OpenStack environment (OS_ variables) just like you do for the native OpenStack clients (typically by sourcing a ~/.ostackrc or ~/novarc or ~/.openrc file). The minimum set is OS_USERNAME, OS_USER_DOMAIN_NAME (could be derived from OS_USERNAME for MyWorkplace users), OS_PASSWORD, OS_PROJECT_NAME (eu-de), and OS_AUTH_URL (https://iam.eu-de.otc.t-systems.com/v3).

Optionally, you can also set some additional defaults on otc_env.sh by editing it and copying it into your home as ~/.otc_env.sh.

When managing multipe OTC domains, you may create ~/.ostackrc.SUFFIX and ~/.s3rc.SUFFIX and set OTC_DOMAIN=SUFFIX to make otc read these files.

Syntax for otc.sh is: otc.sh [--global opts] COMMANDGROUP COMMAND [--command opts] [CMDPOSPARAM]

Please have a look the otc_env.sh file and look at the output from otc.sh help -- this provides sufficient information to get doing. otc.sh also supports otc.sh help COMMANDGROUP to provide only the help section that is specific to the command group.

otc.sh relies on a GNU date, so if you are running it on MacOS X, please install GNU date by running brew install gdate and make it first in the PATH: ln -sf /usr/local/bin/gdate /usr/local/bin/date

Limitations

This tool does not try to cover all functionality of OTC. Rather it tries to be a demo and test tool for many functions; it however does fill in some gaps where the OTC API is extended or different from the standard OpenStack API, so feel free to use it here and contribute to fill gaps you care about. There is a small, but active community on github.com/OpenTelekomCloud/.

The OpenStack REST API has paginated interfaces to allow the consumers to control how large a response may get. Since 0.7.6, otc-tools is using this feature to avoid exceeding the maximum response sizes on OTC. (The API gateway limits responses to a few hundred kB). So otc-tools ends up doing a few REST calls in case you request large amounts of data, such as otc.sh ecs list when you have hundreds of VMs.

You can control API limits using --limit= and --marker= options if you want to manually control the amount of information. You can also use --maxgetkb= to override the default API response limit (250kiB) when relying on autopagination.

The command line parser is not overly smart. The general syntax is: otc.sh [GLOBALOPTS] maincmd subcmd [LOCALOPTS] [POSPARAMS] See otc.sh help for full help and otc.sh maincmd help for help on a specific set of commands.

Note that for most commands that have positional parameters, the command specific options (LOCALOPTS) need to precede the positional parameters.

otc.sh does not do any kind of version discovery, but just assumes support for all commands that public OTC offers. This reduces complexity and increases performance, and is a deliberate choice. It does however use the service catalog from keystone to determine all the endoints.

Some of otc.sh's functions will work with other OpenStack clouds; for testing, even some keystonev2 support has been added. But this is not at all meant to have production quality for any non-FusionSphere clouds.

Output formatting

otc tools tries to output things in a human readable format. It has three basic formats:

• The list format: It creates a table with columns that are separated by 3 spaces. Several values in a field are separated by one space (or a colon for ip:port and for vmid:device). The first three columns are standardized, first is the ID of the resource, second the name, third the status (if reported), then further information (size, AZ, relations, ...) is appended. There can be an optional header (explaining the column contents) which starts with a '#'. This format is returned by list and details subcommands and is supposed to be parsed by humans. (Exception: When new services are implemented, I may just output JSON before I decide on a good list format.)
• The JSON format: The REST calls return JSON objects of course and jq is used to present a nicely formatted represntation of the object. The JSON can be further passed.
• The long running requests report the TASKID. With --nowait, this is the last output. Otherwise, the resource ID will be output last. By default the task status will onlybe queried until the resource ID is known; with --wait, otc.sh will wait for the task to complete. There are some intermediate task status updates and waiting dots in between.

This has been tidied up for otc-tools-0.7 and should be more usable and less fragile now. Nevertheless, this tool is not yet mature enough that we commit to not break the output format here yet -- a future version might well provide switches to control the output. Pull requests are welcome :-)

Custom commands

New in otc-tools-0.7.0, we support custom commands using otc custom GET|PUT|POST|DELETE|HEAD URL JSON You can reference variables by using $BASEURL $CINDER_URL $OS_PROJECT_ID. These are not standardized and well-documented yet (except for the OS_ ones which are the normal OpenStack environment). You'll need to check the source code. But the most important ones you already know now ... If you start the URL with a / (instead of http[s], $BASEURL will be prepended. There is an optional --jqfilter FILTERSTRING. By default, the output from the REST call is passed through jq -r '.'. You can replace this with --jqfilter '' which results in no processing. Or you pass a string like --jqfilter '.servers[] | .id+" "+.name'. Note the quoting for ".

Note that in this mode, otc.sh really does not much more than curl except for handling the tokens and providing the variables like $CINDER_URL from the catalog. The option --paginate ARRNM however might be handy for custom GET. (Current implementation for --paginate is somewhat inflexible in that it assumes records smaller than ~4k and you need to specify the array name from the JSON response. This will improve in a future version.)

Return codes

Some work has been done in the 0.7.x series to tidy up the return of response codes; so otc.sh should really have a non-zero exit code if there were serious failures.

Token cache

otc.sh does cache tokens (and the service catalog) in ~/tmp/.otc.cache.$OS_USERNAME.$OS_USER_DOMAIN_NAME.$OS_PROJECT_NAME (for project-scoped tokens; domain-scoped tokens are without .$OS_PROJECT_NAME). otc.sh ensures these files have safe permission (0600), but if your home directory gets shared or backed up in unsafe ways, you might need to take addtitional caution. Note that tokens are normally valid for 24hrs on OTC. You can use the --nocache global option to not use the token cache. If you have trouble with existing cache entries you might also use --discardcache to force otc.sh to retrieve a new token (and store it).

Note that you can force the usage of domain-scoped tokens using --domainscope, project-scoped tokens using --projectscope and unscoped using --unscoped. otc.sh normally knows what kind of token is needed for which command, but for the custom functions the default (project scoped tokens) may not always fit.

The token cache was introduced in 0.8.0 ot otc-tools.

Packages

Packages (RPMs/DEBs) on open Build Service home:garloff:OTC https://build.opensuse.org/package/show/home:garloff:OTC/otc-tools

A docker container image tsiotc/otc-client is available on dockerhub.

You can of course also just grab this script from github; please install curl and jq to make it work; optionally also s3 (libs3/libs3-4).