Floats for currency in a payments platform.

The system will happily take a transaction for $121.765, and every so often there’s a dispute because one report ran it through round() and another through floor().

A page that handled call requests. It was a table showing some information about the person, the case it’s related to and some other fields. It fetched everything from any table it touched. So the call was fetching all the information about the person. The case it was related to. The person who was assigned to the case, and since the case was linked a couple of layers in, all of that data as well.

I created a simple view that only fetched the data it needed. It went from over A GIGABYTE of data to less than 25mb of data of transfer to the web ui.

This might require a bit of background knowledge about Power Query in Excel and Power BI, specifically the concept Query Folding.

Power Query is a tool to define and run queries against a host of data sources and spit out tabular data for use in Excel (as tables) or Power BI (as Tabular Data Model). The selling point of it is the low-code graphical presentation: You transform the data by adding steps to the query, mostly through the menu ribbon. Change a column type? Click the column header > Data Type > select the new type. Perform a join? Click “Merge Queries”, select the second query, select the respective key column(s) to join on and thr join type – no typing needed. You get a nested table column you can then select which columns to expand or aggregate from.

Each step provides you with a preview of the results, and you can look at, edit, delete or insert earlier steps at will. You can also edit individual steps or the whole query through a code editor, but the appeal is obviously that even non-programmers can use it without needing to code.

Of course, it’s most efficient to have SQL transformations done by the database server already. Bur Power Query can do that too: “Query Folding” is the feature that automatically turns a sequence of Power Query steps into native SQL. A sequence like “Source, Select Columns, Filter Rows, Rename Columns” will quite neatly be converted into the SQL equivalent you’d expect. Merges will become Join, appending tables becomes Union, converting a text to uppercase becomes UPPER and so on.

If at some point there is a step it can’t fold, it will use a native query to load the data up to that point, then do the rest in-memory. Even if later steps were foldable, they’ll have to be done in-memory. You can guess that this creates a lot of potential for optimising longer queries by ensuring as much or it as possible is folded and that the result is as “small” as possible – as few rows and column as feasible etc.

Now, when I tell you that there is a table in one of our sources with a few large text columns you almost never need, you may be able to smell the smoke already. A colleague of mine needed help with his queries being slow to load. He had copied some code from Stackoverflow or what have you that joins a query with itself multiple times to resolve hierarchies. In theory, it was supposed to be foldable, provided the step it runs off of is. The general schema of my colleague’s query went Data Source -> non-foldable type conversion -> copied code -> filtering (ultimately keeping about 20% of rows) -> renaming columns -> removing columns. Want to guess which columns were loaded, processed with each join, explicitly renamed and only then finally understood to be useless and discarded?

“I always do the filtering last, don’t want to miss anything.”

This is your regularly scheduled reminder that MS (and our corporate BI team) can present Power Query as self-service data transformation tool all it wants, that still doesn’t mean it’s actually designed for use by non-data techies.

Not mine, but svn-based JDSL is the best related story that’s always worth sharing.

https://thedailywtf.com/articles/the-inner-json-effect

Disclaimer: this is not really about code, but about using IT in my non-IT workplace and I realized this just yesterday. A rant.

I work in the social sector. Our boss seems to have slipped into position sideways (they did not do our work for a significant amount of time before).

I got zero onboarding when I started working there; everything I know about the organisational ins and outs I learned by asking my colleagues.

The boss seems to actively want to not inform me of things, i.e. even if I ask about something they reply in the most cursory manner or immediately refer me to somebody else. I have no idea why they do it, my guess is that they sense that they’re woefully inadequate for the job, plus me being much older triggers insecurities?

For example, when I could not log into an app to see my future shifts, I asked the boss about it first but they immediately refered me to tech support. Calling them, after a while we found out that the boss had mistyped my name. Then I could log in.

Last week I was sick and waited til Sunday noon to check this week’s shifts - but again I couldn’t log in. The boss answered neither phone nor email. Fair enough I guess, on a sunday. Thankfully tech support was working and after a long while we found out that the app for checking my shifts only allows log-ins from within the workplace network, not the open web.

I almost missed my monday shift because of that. Boss calls me, enraged. I explained the situation. They clearly did not know that the app only allows log-ins from within the workplace network.

All my coleagues tentatively/silently agree that this boss is useless. How do we keep the workplace running, and why is it me who is left in the dark? Turns out they have a Whatsapp group. I don’t use Whatsapp. They asked me repeatedly and urgently to join.

tl;dr: this workplace would fall apart if people wouldn’t communicate through Whatsapp instead of official channels

All about PTC’s God awful piece of shit PLM/PDM systems IntraLink and PDMlink. I cannot believe the amount of trash code that company uses. And they get paid millions to basically screw the customers over. The costumer’s CAD gets intertwined in a huge heap of automated HTML garbage. This leads to a total disaster.

Had a coding firm costing 1k+ euros which was unfamiliar with django select all() from DB just to cast that into a list each time a user opens the tool. That got real funny real fast when the customer started adding the announced 50k objects per day. They did that buried in about 50-60 api endpoints conveniently coded by hand instead of using genetic api endpoints available from django rest framework.

When the loading times hit 50s per click, the company took the money and ran. My colleagues and me spent 2 years and half that to fix that shit.

One time, I had to request firewall access for a machine we were deploying to, and they had an Excel sheet to fill in your request. Not great, I figured, but whatever.

Then I asked who to send the Excel file to and they told me to open a pull request against a Git repo.
And then, with full pride, the guy tells me that they have an Ansible script, which reads the Excel files during deployment and rolls out the firewall rules as specified.

In effect, this meant:

1. Of course, I had specified the values in the wrong format. It was just plaintext fields in that Excel, with no hint as to how to format them.
2. We did have to go back and forth a few times, because their deployment would fail from the wrong format.
3. Every time I changed something, they had to check that I’m not giving myself overly broad access. And because it’s an Excel, they can’t really look at the diff. Every time, they have to open it and then maybe use the Excel version history to know what changed? I have no idea how they actually made that workable.

Yeah, the whole time I was thinking, please just let me edit an Ansible inventory file instead. I get that they have non-technical users, but believe it or not, it does not actually make it simpler, if you expose the same technical fields in a spreadsheet and then still use a pull request workflow and everything…

So, this is completely off topic, but some of the comments here reminded me of it:

An elderly family friend was spending a lot of her time using Photoshop to make whimsy collages and stuff to give as gifts to friends and family.
I discovered that when she wanted to add text to an image, she would type it out in Microsoft Word, print it, scan the printed page, then overlay the resulting image over the background with a 50% opacity.
I showed her the type tool in Photoshop and it blew her mind.

1. Take from index 10 of the buffer, AND it with some hard-coded hex value.

2. Bit shift it by a hard-coded amount of 2

3. Do the first two steps, but with a different hard-coded index, hex value, and bit shift.

4. OR the two results.

5. Shove the result back into a buffer.

All of this is one line with no commenting or references to what the fuck this process comes from or why it is applicable. Then there was a second copy of the line, but with different hard-coded values.

My current favorite is in ruby with the unless keyword:

tax = 0.00
unless not_taxed(billing)
  tax = billing.zipcode.blank? ? estimated_tax_from_ip(account) : billing.tax
  tax = (tax.nil? ? 0.00 : tax)
end

To me, anything payments related you want to be really super clear as to what you’re doing because the consequences of getting it wrong are your income. Instead we have this abomination of a double negative, several turnaries, and no comments.

I found code that calculated a single column in an HTML table. It was “last record created on”.

The algorithm was basically:

foreach account group
  foreach account in each account group
    foreach record in account.records
      if record.date > maxdate
        max = maxdate

It basically loaded every database record (the basic unit of record in this DATA COLLECTION SYSTEM) to find the newest one.

Customers couldn’t understand why the page took a minute to load.

It was easily replaced with a SQL query to get the max and it dropped down to a few ms.

The code was so hilariously stupid I left it commented out in the code so future developers could understand who built what they are maintaining.

I wrote an algorithm that should be recursive but in expediency I wrote a loop that iterates 10 times.

It’s fine but I’m still mad 3 weeks later

Oh, I’ve seen some doozies… The one I remember the most, and I’ve seen this twice, is this:

myClass.TheProperty = myClass.TheProperty;

When I asked about it, the developer said that, well yes, because it reads from one place and sets in another! Not at all difficult to read!

The script I vibe coded to automate part of my job. It’s sloppy and unrefined, but it works, and saves me a ton of effort.