Let's Shelf "Technical Debt" and talk about "Design Debt"

As the Agile philosophy picked up steam (and started generating consulting profits), us developers were introduced to the concept of "Technical Debt."

Technical Debt was intended as a way to present sub-optimal technical implementations in the business "friendly" concept of financial debt.  The delta between the optimal solution and the sub-optimal solution is the debt you burden yourself with immediately.  This debt gains interest as long as you do not pay it back by implementing the (now possibly different) optimal solution.

While this metaphor is somewhat useful when discussing the impact of a decision with management, since it never reaches the stakeholder report, there's (in my experience) very little traction to fully pay back the debt.  You sometime make a "good faith" payment (interest only?), but you're usually just left with something "less bad."

We need to move past "Technical" Debt.  The impact is just not there, and very few engineers (or managers) will champion the payment of something that "just works as it is."  Paying customers pay for new things, or fixing their problems.

I propose we start teasing out the truth of the debt.  One of these is "Design Debt."  Design Debt is a high stakes debt that will crush your pace, and can make a seasoned engineer cry.

Design Debt is the sub-optimal design decision made because you are using an Agile Methodology: requirements firm up over multiple iterations, so the design decision you made in an earlier cycle (sprint, Epic, etc...) is already sub-optimal.

Take, for example, deciding that you should implement an Entity-Attribute-Value data model since you heard the customer make a high level request for a system that you can "dynamically add fields to." 

If you start moving forward with that (because we're "sprinting"), prior to engaging the customer in a deeper discussion, Congratulations!  You've got "Design Debt".

And 3 years later, after you're finally going to production, and find the system cannot show a significant portion of your data because it times out querying for the data, you now have to allocate a full time senior developers time for, oh, 8 months to completely tear it out and put in a normalized data model.

If you get just one thing from the above, make it that Entity-Attribute-Value is a last resort model! 

That Was Quick

I dropped the idea of a streaming service changing the video you're watching for each view, and then AT&T gets called out for deciding to change your stream to better target ads.  I recommend you read the Verge article.

The plan AT&T lays out is much more disturbing: they'll combine data from all their properties - cellular, phone, media  change the stream - which will allow them to track you from watching the ad, to actually acting on the ad.  The Verge describes a customer who watches a car ad, and then is tracked going to the dealership. 

I suppose the positive is that you will not have to fill out that survey question about "Where did you hear about us?"

Delete My Data AND The Extracted Information

While I am generally not pro-active in insuring my privacy (I'm quite beholden to Google services), the idea of a 3rd party deleting my data, yet keeping the information their algorithms extracted from it bothers me.

How would I prove the results they generated were wrong?  What if the results have been collected over months or years, and now the Credit Agency see's me as a high risk?  The data used as input to those algorithms is gone now (yay Privacy & Right to Be Forgotten).  How are you going to prove that it was an error and should be removed?  I'm not even sure what data they were collecting.

Moving forward, what if governments worked this way: your data is only on our systems momentarily, and then removed.  Behind the scenes, you've been tagged as a high risk because of an error in the algorithm, and now you are no longer able to purchase a plane ticket.

How will you correct that situation?

It would be like an e-voting system that "recorded" the vote and then destroyed the actual input/ballot.  It flipped every 3rd or 4th vote, but there's no feasible way to prove and fix it.

So, either keep both my data that you used as input and the information you generated, or keep neither.

Did your Streaming Movie Change?

While I enjoy the convenience of digital streaming - be it Movies or Music - I am struck by the potential for some unknown hand to subtly change the scenes, sounds, dialog/vocals, or even "adjust the message".

Since I've purchased a digital copy only, I would not even be able to check to see if something did change.  Almost like a digital "gaslighting." 

Even if there is a physical, unmodified copy available, very few would be motivated to compare.  If it was truly subtle, it may be difficult to notice it.

I feel that this has been done recently - to a syndicated show in order to change the advertiser - and there being at least some discussion about it.  Unfortunately, I am not able to easily dig up anything on it.

PostgreSQL JDBC + SSL

This serves as a note to myself (and anyone else) on a simple way to connect to a SSL-enable PostgreSQL (9.4+) server using the standard java SSL properties:

Add sslfactory=org.postgresql.ssl.DefaultJavaSSLFactory to your connection string, and configure your JVM javax.net.ssl.* properties as you would for the majority of java applications.

E.g.:

jdbc:postgresql://dbhost:5432/testdb?ssl=true&sslmode=verify-ca&sslfactory=org.postgresql.ssl.DefaultJavaSSLFactory

and, if you use PKCS12 for certificate storage:

java -Djavax.net.ssl.keyStoreType=PKCS12 -Djavax.net.ssl.keyStore=/path/to/cert.p12 -Djavax.net.ssl.keyStorePassword=notapassword -Djavax.net.ssl.keyAlias=mycert -Djavax.net.ssl.trustStore=/path/to/truststore.p12 -Djavax.net.ssl.trustStoreType=PKCS12 -Djavax.net.ssl.trustStorePassword=notapassword

Re-framing Privacy

I really enjoy shows that guide me through various points of history, digging deeper into the day to day minutiae that your history classes in high school and college did not - and generally could not - show us.

You can also find nuggets of knowledge that can expand your understanding of the modern day.  This happened recently while watching Lucy Worsley's "History Of The Home" series - I think it was the "Bedroom" episode.

While discussing the lack of intimacy in the home, she said - and I'm paraphrasing - that privacy was the ability to choose who you share yourself with.  While obvious, and probably not uncommon, the quote rung in my head, echoing through the chambers.  Quick aside: it's worth watching that series just to hear about the origins of "making the bed" and "sleep tight".

It clarified my mistaken presumption of privacy as a passive "something" that you had; it was, in actuality, an action that was controlled by you.  It's what you do, not what you have.

Loss, or invasion, of our privacy, then, is the wrong way to think about the privacy problems we face today in technology, government, and society.   When I hear about it, privacy is presented as a secondary privilege, as if it were my home.

This leads me to believe that privacy should be compared to Free Speech.  In fact, privacy seems to be involved in exercising free speech: I choose to whom, of what, and how much of it, I speak.  Being stripped of privacy prevents me from effectively exercising my free speech rights.  Now the encroacher has not just read your journal, but has identified your personal expressions ("oh.  I see you like to dance as you get into the shower").

In this way, we need to think of this as being stripped of a freedom, rather than a loss or invasion.   Privacy is a choice of what you share, and how much of it you share: you choose when to stop sharing.

Unfortunately, the platforms we use are actively getting in the way of us exercising our privacy: "Look at what is going on out there.  Just take a peek.  Don't you want to say something about it.  Perhaps do something - we can help you do that something."  Kind of like having a kid around: you don't focus on their presence, you don't sense any danger of them being around while your acting out your day, then BAM!  Your kid just told your friend what you bought them for their birthday - or, worse, tells your girlfriend that you bought her an engagement ring (https://money.cnn.com/galleries/2010/technology/1012/gallery.5_data_breaches/3.html). 

Here's a thought: when you go into a store, you have a couple areas where you could exercise privacy (e.g. the bathroom).  Where can you do that on Facebook?  On any Google property?  Or even Amazon or Apple?  Every move you make goes directly into their internal data set, which they can parse whenever they want (in private too!)

This is definitely a topic to revisit soon.

Agile Development: The Psychological Toll

I have a new hypothesis about Agile Software Development: it will cause a form of "Stockholm Syndrome" along with a healthy serving of Cognitive Dissonance in a significant number of developers.  Many of those developers become ardent evangelists that will not even consider alternatives - or even deviations - from their Agile Methodology.

I've been working on or with teams using Agile methodologies for over a decade.  It will be around for a while.  Yet, there has always been a problem with those teams...there was always a feeling of tension at the end of the sprint, even though it was not acted upon by the developers; it was internalized as shame, resentment, self-loathing, and even a mild depression. 

In the last 2-3 months, some on my team, and myself have waddled through such feelings.  I'm fortunate to have been able to work through them, and to emerge with a bit more understanding of why I've had such a bias against Agile.

With a team using Agile Methodologies, there is an overwhelming urgency to complete your selected story/task/bug/issue within the sprint cycle - "that's what the mantra's say!  That's how Agile works!  Everyone else who used Agile does it, and does it better than me/us!  I cannot be the slacker!"

And then it happens. 

You've gotten a few sprints closed.  You completed your tasks in time; some were close, but it was probably just the extra cycles spent on Reddit.   Staying an extra couple hours made sure I kept velocity at the right level. 

The next sprint was not so fortuitous.  You did not finish a task.  It was only 5 story points.  I'm an idiot.  Don't worry, I'll finish it up during the first few days of the next sprint, and still complete my typical number of points.   

I'll just work a few extra (unpaid or unreported) hours.

Now you've moved into the mindset of "keeping up with the team," rather than "solve a problem effectively for this domain/customer".

We're coming up on the end of the next sprint.  I've done a decent amount of work creating and coding a solution...but it feels wrong.  I could really use a day to review and maybe (no...no!) rework the solution.   

Nope.  No reworking.  What I have implements the fix/feature.  It'll fly through testing. 

But there's something wrong with it...I really need to know...
Ugh.  Close the ticket.  Now ignore it.

There it is.  The start - or perhaps relapse - into a state of Cognitive Dissonance: You believe deeply that you are an excellent Software Engineer; you create solutions for problems that exceed and amaze not only your customers, but your team and even yourself.  Yet you have just committed a solution to the baseline that you feel is...wrong...insufficient for the actual problem as a whole.

The show must go on, so you have to convince yourself that it must be the correct solution.  Agile is good, right.  It knows something you don't.  Master it and you will become the master.   

And if others follow it, they will see me as a master....  If others follow it, I am not wrong.