./ Python AWS Shell Cockpit

$:~ pyawscp

Overview
Quick Start
Docs
Versions
Notes
Contact

A simple "Python Shell" environment that you can use to "talk" with your   account...

--> Install and enter...
$:~pyawscp


--> Graphically navigate on your AWS Networking resources using the command
$:~navigator
# Notice! Navigator will likely not work on Internet Explorer


--> Export to drawIO the views generated by the command
$:~navigator


--> Upload objects in parts to S3, using the Multipart Upload command
$:~transferS3 my-bucket-name,/home/myself/my-big-file.tar.gz


--> Generate a "traditional" AWS Networking Diagram (VPCs, Subnets, CIDRBlocks, RTs, NACLs, etc.) at DrawIo, using CVS Import.
$:~drawNetworking all


--> Search for which EC2/ELB are behind a Route 53 DNS (view in Graph or ASCII)
$:~nslookupEc2R53 www.my-company-site.uat.com,[graph]


--> From the results, generate Excel spreadsheets and DrawIo Diagrams with the parameters "excel" and "drawio"
$:~ listSg list-associated list-permissions excel drawio


Quick Start

--> Install & Run...


#
# As a Python application, to install enter...
$ pip install pyawscp


#
# Wait! Before starting it...
# as minimum configuration, your AWS Credentials must be setup
$ cat ~/.aws/credentials
> [default]
> aws_access_key_id = YOUR_ACCESS_KEY
> aws_secret_access_key = YOUR_SECRET_KEY

# And optionally...
$ cat ~/.aws/config
> [default]
> region=us-east-1


#
# To start...
$ pyawscp


Docs

All the documentation is available inside the own PyAwsCp Shell environment, check it out:


#
# Once inside, list all available commands...
(aws-shell-cockpit)⛅:~ ls


# or...
(aws-shell-cockpit)⛅:~ ls -l   # for a long and detailed listing of all available commands


#
# View how to use and check all options available for a specific command:
(aws-shell-cockpit)⛅:~ help [command]
(aws-shell-cockpit)⛅:~ help listEc2
(aws-shell-cockpit)⛅:~ help nslookupEc2R53
(aws-shell-cockpit)⛅:~ help transferToS3


#
# In addition, you can also...
(aws-shell-cockpit)⛅:~ clear
(aws-shell-cockpit)⛅:~ !ls or !dir  # Run a command on your OS
(aws-shell-cockpit)⛅:~ [TAB]        # Auto complete
(aws-shell-cockpit)⛅:~ [Arrow-Up]   # Command history



######################################################################
# More things...
# Almost every function works with "generics" arguments that
# makes possible some customizations at the result:
######################################################################
# desc           - List in descending order (use the first column as default)
# sort3          - Sort the table by a specific column number
# showtags       - When applicable, show the tags of the resource
# verbose        - Before the result, shows the same content (or more) in JSON
# save           - Save the same result's content in a external JSON file (results folder)
# | {value}      - Highligth the lines with the {value} content
# | grep {value} - Filter the lines with the {value} content
# Examples:
(aws-shell-cockpit)⛅:~ listEc2 sort4,desc
(aws-shell-cockpit)⛅:~ listEc2 | grep uat
(aws-shell-cockpit)⛅:~ listTargetGroups verbose
(aws-shell-cockpit)⛅:~ listBucketsS3 save,verbose


# And...
 # - After every executed command, the result is also copied to your Clipboard!
# - At navigator, use the mouse-right-button to opened permanently the window's popup info

Versions

  • v0.6.11 (Oct 8, 2021):
    •  Management and persistence of Role, MFA and Profile
    •  Import data from AwsSee
    •  Organized functions to manage AWS Session info
  • v0.6.10 (Oct 4, 2021):
    •  Configure the thread reuse for the local Navigator HTTP Thread Server
  • v0.6.9 (Oct 4, 2021):
    •  Enable to open Navigator from WSL Windows (linux) using Windows Browser
  • v0.6.8 (Oct 4, 2021):
    •  List the configured AWS credentials
  • v0.6.7 (Aug 27, 2021):
    •  Allow assume role without need a MFA set
  • v0.6.5 (Apr 09, 2021):
    •   Function groupByEc2: It performs an aggregate function over EC2 Instances based in one of its attributes
    •   Function findRouteEc2: Given an Instance Id, it will trace all routes to reach to it using private/public Route 53 records, through ELB or IP Address
  • v0.6.4 (Mar 30, 2021):
    •  Fixed feature hightlighting/filtering lines in tables views.
  • v0.6.3 (Mar 29, 2021):
    •  Fixed DST problems regarding token expiration (MFA Session).
  • v0.6.2 (Mar 11, 2021):
    •  Forcing update boto3 library to a specific minimum version (>=1.14.2) during installation.
  • v0.6.1 (Mar 05, 2021):
    •  Falling to start, attempt to load a module not required for this version, experimental code removed.
  • v0.6.0 (Mar 04, 2021):
    •   Generate an Excel spreadsheet from a result
    •   Generate a Graphic Diagram for DrawIo using CSV (Arrange --> Insert --> Advanced --> CSV...) - available for some commands
    •   Generate an AWS Network Diagram for DrawIo using CSV (VPC, Subnets, Route Table, NACLs, Security Groups, Transit Gateways)
    •   Generate an PreSigned URL for an object in S3 Bucket
    •   Allow to Assume Role (in cases like: access another Account, or MFA is activated for the AWS CLI Credentials being used, STS Tokens)
    •   Use of MFA for AWS CLI Credentials (STS Tokens)
    •   New commands available: List Security Groups, List Route Tables, List NACLs, Role Info and LogOff
  • v0.5.32 (Sept 23, 2020):
    •  List EC2 crashing when there's an EC2 Instance "still" in terminated state.
  • v0.5.31 (Sept 18, 2020):
    •  Navigator crashing when no Tags detected on Ec2 Instances.
  • v0.5.30 (Sept 18, 2020):
    •   Still lot of things to do, but let's share it anyway. There are 17 functions available.


Configuration

The PyAwsCp Shell offers some options to configure preferences on your environment, to check which one, use the command:


#
# Show your environment preferences...
(aws-shell-cockpit)⛅:~ environment


# or...
(aws-shell-cockpit)⛅:~ env   # The short form

# You will get something like this...
 
 
# 
# Each configurable option show its value and explains also how to change it, examples:
(aws-shell-cockpit)⛅:~ set threshold 256  # Change the limit when perform Multipart Uploads
(aws-shell-cockpit)⛅:~ set profile my-account  # Change profile AWS Credential to use
(aws-shell-cockpit)⛅:~ get threshold           # You can also ask for the current value set
❖ Multipart Upload Threshold is set to 256
 

Notes

###  What is this about?
This tool is a simple result, a "collateral-effect", born from the need to perform some eventual boring and repetitive tasks during interaction with AWS, while I was working on my duties.

It's a Shell that contains a collection of pre-built commands that basically extracts information: ASCII Views and also Visual Graphics from the AWS resources and, sometimes, their relationships as well.

In a ordinary working day with Cloud, questions like below always pop up (frequently and recurrently):

  • Which ELB is pointing to this EC2 Instance? Actually, is there any one?
  • Which ELB or EC2 Instance it is the target of the DNS myproject.lof.middle.earth.org ?
  • Is this Subnet is Public or Private after all?
  • How many IP Addresses still left in all my VPCs?   Or... only tell me for this one here vpc-0123456789abcdef?
  • Is there any S3 Multipart Uploads unfinished?   So, let's abort it... (this costs money and I cannot see them on AWS Console)
  • Give me a List of all my VPC's Subnet, tell me also which one is Public


###  So What?
So, we could just use AWS Console, AWS CLI + Bash Script, Boto3, right? Well, pretty much it is what I was doing all the day around. But, I realized that:

  • Interact with AWS Console it's counterproductive (boring, slow, ...)
  • Manage to get the answers with AWS CLI (although more flexible than AWS Console), very often you have to run two, three commands to reach to the final answer.
  • Besides, using AWS CLI, you have to mantain the used commands saved in some place (well commented) and available, like your library.

As Python Boto3 is far most powerful than AWS CLI, it can offers you tons of features to interact with AWS (it's not by chance that tools like Ansible Red Hat, after all, use it). I came up with the idea of build this sort-of "Shell" AWS utility to my day-to-day tasks, to help me, make the repetition less boring and bring some agility as well. That's where was born this idea (well, also something to do on my spare time when I am bored)  

All the commands (that were and still are useful to me) are pre-packaged inside this Python Shell, now I only need to install the Python package (running a pip command) and have everything available and documented to use it.


###  Then...
While working on this, some other ideas were popping up, like:

  • Generate some visual graphs (exportable to DrawIO) as vision of the AWS resources and their relationships.
  • An online navigator, where you can view (graphically and export also) your AWS Networking resources (online - a live view).

Some other things I was thinking of, backlog:

  • Generate a durable history of commands, that is, after exit when you come back all of the past N commands entered, will be available to relaunch
  • A way to save a command with all its inputs / arguments / preferences. Like a macro, to recover later and re-launch the same command, exactly as it was.


I am always trying to add new features, when new needs or ideas are raising (of course when I have some off-time to dedicate to it).

I am sharing the result of this, perhaps can be useful to someone else. Feel free to reach out to me with new ideas/suggestions, I will be glad to hear you.



Contact

   http://linkedin.com/in/ualter
   ualter.junior@gmail.com
   paypal.me/ualter --> just in case you'd like to pay me a coffee for the work