Subset sum problem in Ruby

I came across a bizarre data storage decision in a recent data migration. For context, in Australia there is a kind of government demographic survey that must be reported to by certain organisations. One of the data points is “Qualifications Achieved” or something to that affect, which accepts a comma-separated list of values. For example, the qualifications and their values are similar to:

524 - Certificate I
521 - Certificate II
514 - Certificate III
410 - Advanced Diploma
008 - Bachelor Degree

If a person had achieved a Certificate III and a Bachelor, you would report 514,008 for that person to the government, for that data point. In the database in question there was a column which stored a single value. In this case it was 522, which is 514 + 008. So, if I wanted to break apart this number into its component parts to store it a bit more sensibly, I needed to figure out which of the source numbers added up to the target number.

I’m sure any developer reading this has had a problem where they are sure there is an answer, but they just don’t know what to search for. After some Googling it turns out this is called the subset sum problem. And someone had thoughtfully made an implementation in ruby which I could use:

http://ruby-subsetsum.jeremyevans.net

Note that in my case I needed only one output set, which worked because all the number combinations in my source set of numbers provide a unique result. E.g. for the numbers above no combination except 514 + 008 adds up to 522. If you need it to this algorithm also returns multiple number sets that add up to the total.

So, I took the algorithm, took my source numbers for each different data point, and my totals from the database, and it spat out the correct combinations! 1053 = 008 + 521 + 524. Aren’t algorithms magic sometimes?

Global rescue_from error in Rails application_controller

In our (Webbernet at time of writing) rails application, we needed a way to raise security access violations based on the user profile and bubble them all the way up to the application controller. We looked into it and found you can use rescue_from in your application controller, which allows you to specify an error class and a method inside the controller to call when that error is encountered inside controller actions and filters. For example:

class ApplicationController < ActionController::Base
  rescue_from Errors::SomeCustomErrorClass, with: :handle_error_method

  def handle_error_method(error)
    # Do some error handling
    redirect_to root_path, alert: error.message
  end
end

It’s probably not really a good idea to handle the normal ruby StandardError in this way, as that may get you into trouble, but it is perfect for custom errors raised deliberately from within your application! I really like this pattern of nesting an error definition class inside the class that is the one to raise that error. For example, in the result of a security check:

class SecurityCheckResult
  class AuthorizationError < StandardError
  end

  def run
    raise AuthorizationError, "User is not authorized" if check_invalid?
  end
end

Then in application controller I could just rescue_from SecurityCheckResult::AuthorizationError to catch this anywhere in my app, and do something like a redirect or a flash. If you need to use this pattern in regular ruby code you can include the ActiveSupport::Rescuable module. This article has a great example of using the module in regular ruby code (scroll down to the part that mentions RoboDomain).

Alternatively, you can look at the rails documentation for Rescuable

Find duplicate rows in SQL

Sometimes you need to find and count duplicate data rows in SQL. For example, in my use case I needed to find records in a table where there was more than one usage of the same email address. This would help me figure out how widespread and severe the duplicate issue was; the table in question should not have had duplicate rows based on that column in the first place! (A missing UNIQUE index was the culprit).

SELECT email, COUNT(*)
FROM user_accounts
GROUP BY email
HAVING COUNT(*) > 1;

The HAVING clause is the important part of this query. To find duplicates, we need to check if any of the groups have a record count > 1. You can put other conditions for the groups in the HAVING clause as well if required, e.g. COUNT(*) > 1 AND account_status = 1.

The result of this query can then be used for a sub query/WHERE clause. The result looks like:

email              | count
--------------------------------
j.wayne@gmail.com  | 2
g.cooper@gmail.com | 3

Getting nodejs file permissions from fs.stat or fsPromises.stat mode

When you need to get file stats using NodeJS (which calls the unix stat command in the background), you can use the fsPromises.stat call as shown below:

import { stat } from 'node:fs/promises';
const stats = stat('path/to/file');

The stats object returned here is an instance of fs.Stats which contains a mode property. You can use this property to determine the unix file permissions for the file path provided. The only problem is that this mode property just gives you a number (as referenced in this GitHub issue). To view the permissions in the standard unix octal format (e.g. 0445, 0777 etc) you can use the following code:

const unixFilePermissions = '0' + (stats.mode & parseInt('777', 8)).toString(8);

Some examples of the mode before and after calling the above snippet:

33188 -> 0644
33261 -> 0755

field_with_errors changes page appearance in Rails

It seems like it’s easier to do the following to disable the behaviour entirely. Create an initializer file named config/initializers/field_with_errors.rb:

ActionView::Base.field_error_proc = proc do |html_tag, instance|
  html_tag.html_safe
end

I had a minor issue with my Rails view when I had a list of radio buttons wrapped in labels. When there are form errors on a field like a radio button, Rails puts the CSS class .field_with_errors on that field. This causes some issues with alignment as seen in the screenshot below:

All you need to do to fix this is make the .field_with_errors class display inline like so:

.field_with_errors { display: inline; }

Set timezone from terminal OSX

I often have to switch between timezones to test our timezone-sensitive application code. I was getting annoyed at having to open the settings screen in preferences (which is slow) and found out how to do it from the command line.

To set your timezone run:

sudo systemsetup -settimezone timezone

Where timezone is a valid zone from this list:

sudo systemsetup -listtimezones

Finally, you can get your current system timezone using:

sudo systemsetup -gettimezone

This command can easily be made into an alias like so:

settz="sudo systemsetup -settimezone $@"

So all you need to do to change your timezone is settz GMT!

Invalid byte sequence in US-ASCII

After some new code was checked in at work (in 2017 this was Webbernet) we encountered this issue in our CI as part of the build step to run RubyCritic over our code. I’d never seen it before, and the source of the error was in buffer.rb of the parser gem library:

'source=': invalid byte sequence in US-ASCII (EncodingError)

I did some digging and I found that this is where RubyCritic parses each file into an abstract syntax tree for analysis. It seemed like there was a character in the file that could not be parsed correctly, and eventually I found a StackOverflow post that pointed to a tool called iconv that can be used to convert between different character encodings, and that if a conversion is unsuccessful it will throw an error and return code 1. Now this was all well and good but the error I was getting from buffer.rb did not tell me the currently erroring file – the best I could do was modify my local gem source to give me a list of the files that passed through the RubyCritic library for analysis.

Then, now that I had a list of files, I could run each file through iconv to check which one had invalid ASCII characters. Of course I am a programmer and thus lazy so I wasn’t going to sit there and run it manually on every damn file, so I just made a ruby script to run it on each file in my list (of which there were hundreds):

def run 
  SOURCE_FILES.each do |file|
    file_path = SOURCE_DIR # source dir is the full path of the root directory
    puts file_path
    puts `iconv -f us-ascii #{file_path} > /dev/null; echo $`
  end
end

I ran the script and it found the file easily by finding the one that returned 1. Then, all I did to fix the issue was delete the code that had been changed in the previous commit, re-typed it manually, then saved the file. I ran my script again and the issue was solved!

Expanded output format for PSQL

If you are using psql you may be getting annoyed that your query results look like this for tables with more than one or two columns:

Well, there is an answer to this problem. Just enter the command \x on and you will turn on the expanded display option, which makes your query results look like this:

Much better!