irldexter

Friday, April 04, 2014

One liners

So I've always been a fan of the unix philosophy with a passion for trying to do most things in one line of/with Bash rather than a full script or program. I am by no means adept but battle through with sed, awk, paste, cut, tr, sort etc... So when one can potentially use curl on a RESTful service, and stay on the command line rather than logging in to a web app, I'll give it a go.

We use Saasu for our back-end invoicing and reconciliation and they provide access to their API via a secret key (tied to configurable users). I decided I wanted to see what we were owed and owing via the command line so with the help of some other simple tools I came up with the below. It's still a work in progress and I'm open to all the help and any suggestions I can get ;)

First you may need to ensure you have 'xmlstarlet', 'dialog', and 'cURL' installed on your *nix system via ports, apt, or otherwise. Ensure they are happily found in your $PATH. Then replace the below with your Saasu secret/access key 'XXXXXXXXXXXXXXXX' (preferably from a read only user account) and the file ID 'YYYYY' of your desired Saasu account. You can find out how to enable the web services API from Saasu here.

Note: I actually ended up putting the below one liners in respective files and calling them (but you can alias it just as easily... I just didn't want to have to keep sourcing it while testing/editing)... and bingo you have a command (you can call whatever you like).

Owed (the following is one single line):
dialog --title "Company Accounts Receivable" --msgbox "`echo -e "\r\n\r\n" && curl -s "https://secure.saasu.com/webservices/rest/r1/invoicelist?wsaccesskey=XXXXXXXXXXXXXXXXXXXX&FileUid=YYYYY&&transactiontype=s&PaidStatus=unpaid"  | xmlstarlet sel -t -m //invoiceListItem -o "Invoice #" -v invoiceNumber -o " is/was due by " -v dueDate -o " for " -v amountOwed -o " " -v ccy -o " by " -v contactOrganisationName -n | sort -n -k2 && echo -e "\r\n\r\n"`" 60 100 ; clear

Owing (the following is one single line):
dialog --title "Company Accounts Payable" --msgbox "`echo -e "\r\n\r\n" && curl -s "https://secure.saasu.com/webservices/rest/r1/invoicelist?wsaccesskey=XXXXXXXXXXXXXXXXXXXX&FileUid=YYYYY&&transactiontype=p&PaidStatus=unpaid"  | xmlstarlet sel -t -m //invoiceListItem -o "Invoice #" -v invoiceNumber -o " is/was due by " -v dueDate -o " for " -v amountOwed -o " " -v ccy -o " to " -v contactOrganisationName -n | sort -n -k2 && echo -e "\r\n\r\n"`" 60 100 ; clear

So now I just type 'owed' or 'owing' on the command line to get:


I know there's a lot more that could be done, tweaked, improved, and extended... so let me know what you're thinking via @irldexter on twitter if you want to get in touch!



Thursday, March 27, 2014

Changing Masks

So a customer reckons a quick subnet mask change on their router will increase their available host range on their management network.. just like that... :(

Problem: Why increasing a subnet mask (even when you can keep the same gateway) breaks things if you don't update all the associated nodes masks (and additional assets that reference that new increased subnet)... can you help with, add to, or validate the list of issues below?

Example / Details :
  • Router is a Cisco 5548 running NX-OS 5.0(3)N1(1c) 
  • Original network: 10.4.66/24 Router gateway: 10.4.66.1 Router mask updated: to /23 i.e. 255.255.254.0 (The 10.4.67/24 is free for use).
  • Updated network: 10.4.66/23 Router gateway remains: 10.4.66.1 
  • Existing endpoints not updated and remain with original /24 mask. 
  • Only new endpoints in the higher portion of the habitable range and the router now have /23 mask
  • VLAN ID stays the same. 
  • Network consists mainly of management servers and infrastructure devices management interfaces. 
  a) If a host A with its old /24 mask tries to talk to a new host B in the higher portion of the router updated /23 (i.e. 10.0.67.x ) it can not do so directly via ARP. It assumes the other host is on a remote subnet after checking its own host A mask. The A host (thinking it's still on a /24) then sends all traffic to the default gateway rather than via local means to host B. The router has to then process the frame/packet, do a lookup, and forward to host B essentially doubly handling a frame and packet rather than a conversation which could have remained fully on the local switched fabric.
b) Any hosts with static routes configured for the initial /24 will follow their default gateways to reach the new /23 higher portion without having their static routes updated. This may affect multihomed hosts with multiple egress interfaces that require the non-default gateway to communicate to remote management networks for example.
c) The new network will have to be confirmed as being advertised in all required infrastructure routers routing tables, VRFs, and associated statics. (Equally this may affect VPN concentrators, layer 3 switches, or any devices that perform either dynamic or static routing).
d) Any NAT rules will have to be updated to allow for the new /23 and associated pool sizes and mappings.
e) Any firewall objects/network objects will have to be updated to reflect the new network size.
f) For any hosts that use the IP broadcast address to communicate (as opposed to the layer 2 all hosts broadcast address of ff:ff:ff:ff:ff:ff ), the /24 broadcast address is 10.4.66.255 whereas the /23 broadcast address is 10.4.67.255… (albeit 255.255.255.255 will always reach either) thus all endpoints/hosts should be updated.
g) Methods that use proxy ARP or (potentially gratuitous ARP) from the /24 range may fail to update the router and/or hosts if the IP is not deemed to be from the correct subnet.
h) any infrastructure/router ACL(Access Control Lists) that reference a /24 mask will now have to be updated to reflect the /23 mask or connectivity/reachability may suffer.
i) any infrastructure/router prefix lists, policy maps, or traffic engineering that references these subnets or utilises the ACL's above may fail without being correctly updated.
j) if one was to update required endpoints/servers with the updated /23 mask many devices may cache the old mask and/or require a networking restart or route flush before performing.



Tuesday, February 18, 2014

Working Patterns

Problem statement: María is on online Spanish teacher and she was trying to figure out the best weekly schedule to be available for her European students including when she could happily block out personal time (without losing valuable business). She had a gut feel about her busy times of the week but had done no objective analysis up until now. With a few years of data in her work Google Calendar we set about looking for plugins that would let us visualise her working patterns but to no avail.

If you use Google Calendar for your work appointments there's a quick hack below to get a distribution of your busiest times of the week we have come up with. It's not fully automated or too fine grained but hey... we'd welcome any suggestions/improvements.. (there's probably tonnes of prettier, faster, shorter and more accurate programmatic ways to do this!).

Step 1. Download your private "ICAL" .ics file via clicking your private calendar ICAL icon under calendar settings/details and then right click the URL to save the .ics file locally on your machine.

Step 2. Run this bash one liner below on a linux box (as you need to use the coreutils date for the arguments used):
grep "DTSTART:" yourcalendar_file.ics | cut -d ':' -f2 | awk '{print substr($0,1,4)"-"substr($0,5,2)"-"substr($0,7,2)" "substr($0,10,2)":"substr($0,12,2)":00"}'|while read line; do date --date="$line" "+%a %H"; done | sort | uniq -c | sort -k2,2 -k3 > results.txt
Step 3.  Manually enter the results in https://infogr.am/ using a stacked column chart to achieve the graph below!





Saturday, February 08, 2014

On Networks

A brief collection of some interesting and engaging talks on 'networkism' which happens to fit my worldview as a network engineer and zen buddhist informed permie! This was a list I put together on request from someone who was interested in delving deeper in to the burgeoning synthetic superorganism we are building (as opposed to perhaps the existing one!). These talks were chosen for their entertainment value yet deeply profound implications...  (albeit not elevating my own talk to this level, hope you enjoy the collection ;)

RSA animate: The Power of Networks : 2012



Nicholas Christakis : Ted : The Hidden Influence of Social Networks



Kevin Kelly : LinuxConf : The Technium : 2013



Alexander Bard : Tedx : From Relativisim to Networkism : 2013

My beginnings of a synthesis : ZIP : (prep for a talk I gave) : 2009



Thursday, October 24, 2013

Codified

In my experience, there's an inherent problem with having to codify any anti-discrimination, anti-harassment, or general conduct policies. It's not just the inherent challenges of any form of whitelisting or blacklisting but rather having to do so in the first place is already and unfortunately an admission of failure. From here on in one must take a deep and hard look at the problem, its roots, and all the subtleties and sophistication needed to tackle it. If the issue has already arisen due to a discrepancy in shared expectations, morals, or values - then retrospective corrective action in of itself is not enough. Ambiguous and generalised signposts are not enough. Some hard hitting explicit and clear action is required as somehow society and the community has bred this cancer already. The sample space is that of complicated and confused free radicals (i.e. humans), and in this case it seems... not very enlightened, mature, or disciplined individuals and tribes.

And when the bar has fallen too low it must be reset with painstaking detail, clarity, and at an extremely high level to ensure the laggards have no shadows left to intentionally or unintentionally lurk in.

There will always be outliers, some bad behaviour, or even crimes committed in any sufficiently large gathering of people yet an abnormal statistical frequency or growing stereotype is indicative of a much deeper problem. This also highlights that existing laws, conventions, or protocols are either not well known, adhered to, or not sufficiently and deeply realised... for if they were, there would already be an operating Minimal Viable Behaviour that is self-policed (and in many cases there is). Here judgement is implied and the capability for self-judgement and self-doubt is crucial. Additionally, this faculty must remain active and unimpaired at all times to ensure correct engagement with oneself and others. In some cases people just don't care about their bad behaviour nor can they recognise it - and as such they must be reprimanded, potentially excluded, and indeed educated.

Policies serve many purposes, including but not limited to, providing a signpost, a guide in setting shared expectations, an implied agreement that can be retrospectively held aloft to demonstrate an explicit breach of protocol (after the fact)... however policies are worthless if they can not be enforced, can be easily gamed or invalidated, or are not believed to be applicable to the parties they are actually intended for... and herein lies one of the fundamental problems.

Many may argue for concepts of common sense or basic morality however I believe such things not to exist. There are reasons for altruistic and non-violent behaviour - for example; in supporting social cohesion and basic survival, however when there is aggressive or violent speech or action (and for true progress to be made), the root cause must be addressed and not solely corrected in a reactive manner. There are a multitude of life trajectories that individuals experience and many paths are indeed partially or collectively shared, yet the intersections are not actually and in any way universal (other than perhaps birth, death, hunger, cold, and most physical sensations etc). Subsequent higher order cognisant overlaps are in fact rarer than one might think especially when ones unique consciousness and relationship with meaning or understanding is considered. In the same way that it is difficult to guarantee anything but a tiny common vocabulary to begin with - meanings, subtleties of expression, experiences, and shared understanding of concepts can thus vary widely. It is in this vein that I submit there is no universally shared secular morals, ethics, or commonly accepted and subsequently codified set of acceptable and readily understood behaviours. One exception may be that of a tiny common framework of concepts such as 'thou shalt not kill' or the Golden or Silver Rules of which they themselves are dependent upon self-referential interpretations and subjective application. In this context, and explicitly at an individual and universal level, there is no equivalent of the UNDHR(United Nations Declaration of Human Rights) other than some corpus of local laws that are not in themselves obvious or easily recounted.

So I challenge all sexes in the technology industry; it's time to step up our game, refine the problem statement, and actually push things forward whilst earning our innovation moniker. I'd also like to stress that sometimes what's innovative is actually going back to basics and making a list of actionable items:

a) over and above ambiguous 'catch-all' policies - define the most common breaches of human protocol at gatherings with examples. This is not for the 99 but the 1% of offenders who don't understand what behaviour is unacceptable, borderline, or sadly lack the self-referential judgement and empathy required. Some lack the ability to contextualise the impact of even their most 'innocuous' actions. If it's time to be more explicit (which it already is), BE MORE EXPLICIT whilst keeping your lists and catch-all terms like 'including but not limited to'.
b) make explicit opening statements at the main events and keynotes rather than just 'darkweb' documentation
c) put up signs and warnings (yes, akin to McDonalds 'Contents are hot' which is already a societal fail for most because we're also past that point already, time to wake up and smell it)..
d) consult and solicit feedback from known victims
e) give victims both an anonymous and attribution based feedback mechanism
f) pay more attention to how your policies can be gamed and defend against such
g) consider eliminating whole classes of problems until the situation improves across the board (it's already desperate times, desperate measures may serve temporarily to highlight the problems)
h) consider having attendees accept Terms and Conditions or Acceptable Attendee Behaviour when signing up (if it's good enough for software, it's good enough for humans)
i) reinforce that illegal behaviour is not tolerated and will be reported to the authorities whereupon the organisation will also seek prosecutions
j) look to other industries who are making strides in the problem space

Unfortunately many do not understand what it's like to actually be a victim until you have actually been a victim (whether of bullying, abuse, or violence etc.) so err on the side of caution. Accept the problem is already a serious deficit of 'good' judgement regarding what's acceptable, respectful, and legal. Spare a moment to consider that the leaders of the community drafting the policies are probably the least likely to personally experience abuse or attacks, the least likely to inhabit a consciousness that commits abusive acts, and perhaps the least likely to want to believe their community is even capable of such acts.

I posit 3 fundamental and supplementary 'back to basics' guidelines or simple rules/protocols:
1. Respect and don't breach anyone's personal space ( minimum > 0.5m )
  1.1 Touch is the ultimate breach of personal space. No touching.
2. No individual or group insults, slurs, hate speech or derogatory comments related but not limited to race, religion, gender, or appearance.
3. Practice non-aggression, non-violence, and harm minimisation with regards to all thoughts, speech, and action whether directed externally or internally.



Thursday, October 03, 2013

Sakura

Physically she's far away but hiding inside.
A bud and flower in one, a power laying consciously dormant yet blooming unconscious in flow,
An edge of introspection and sharp wit,
A row boat adrift with ample oars,
Willing, able, capable,
Smart, sexy, funny,
Deep, quiet, restrained,
Ready,
Not just to find a new voice, but to roar because…



Tuesday, April 02, 2013

ThirdSpace

Both a bridger and a builder,
Straddling the system of Mu.
Layers in all directions,
Intersections and interfaces hold volume.
Longest not shortest path first,
No need for a salesman to travel.



Twitter'ish musings...

    Come join me on Twitter