Technical Explorations

Enforcing Subclass Method Implementation in WiFi Wand Using TracePoint

2025-08-14T00:00:00+00:00

Ruby’s `inherited` Hook: The Timing Problem and a TracePoint Solution in WiFi Wand

When building cross-platform Ruby libraries like WiFi Wand, you’ll likely encounter the inherited hook. This powerful callback allows parent classes to respond when they’re subclassed. However, there’s a subtle timing issue that can trip up even experienced Ruby developers: the inherited hook fires before the subclass is fully defined.

The Problem: Early Execution of `inherited` in WiFi Wand

Let’s say you’re building WiFi Wand’s architecture where you want to automatically verify that OS-specific model classes implement all required underscore-prefixed methods. Your first instinct might be to use the inherited hook:

class BaseModel
  UNDERSCORE_PREFIXED_METHODS = %i[
    _available_network_names
    _connected_network_name
    _disconnect
    _ip_address
  ].freeze

  def self.inherited(subclass)
    puts "Verifying underscore methods for #{subclass}..."
    missing_methods = UNDERSCORE_PREFIXED_METHODS - subclass.public_instance_methods
    unless missing_methods.empty?
      raise NotImplementedError, "Subclass #{subclass.name} must implement #{missing_methods.inspect}"
    end
    puts "All underscore methods implemented!"
  end
end

class UbuntuModel < BaseModel
  def _available_network_names
    # Implementation that scans for available networks
  end
  
  def _connected_network_name
    # Implementation that gets current connection
  end
  
  def _disconnect
    # Implementation that disconnects from current network
  end
end

Running this code produces surprising output:

Verifying underscore methods for UbuntuModel...
NotImplementedError: Subclass UbuntuModel must implement [:_available_network_names, :_connected_network_name, :_disconnect, :_ip_address]

The missing methods list includes methods that are actually defined in the subclass! But why?

Understanding the Timing Issue

The inherited hook is called immediately when Ruby encounters the class declaration line (class UbuntuModel < BaseModel), but before any of the method definitions in the class body are processed. Here’s the execution order:

Ruby sees class UbuntuModel < BaseModel
inherited(UbuntuModel) is called immediately
The class body is processed (method definitions, etc.)
Class definition completes

This timing makes inherited perfect for setting up class-level configuration or establishing inheritance hierarchies, but useless for inspecting a fully-defined subclass.

Let’s prove this with a more detailed example:

class BaseModel
  UNDERSCORE_PREFIXED_METHODS = %i[_available_network_names _connected_network_name _disconnect].freeze

  def self.inherited(subclass)
    puts "inherited called for #{subclass}"
    puts "Methods at this point: #{subclass.public_instance_methods(false)}"
    missing = UNDERSCORE_PREFIXED_METHODS - subclass.public_instance_methods(false)
    puts "Missing methods: #{missing}"
  end
end

class UbuntuModel < BaseModel
  puts "About to define _available_network_names method"
  
  def _available_network_names
    `nmcli -t -f SSID dev wifi list`.split("\n")
  end
  
  puts "About to define _connected_network_name method"
  
  def _connected_network_name
    `nmcli -t -f NAME,TYPE connection show --active | grep 802-11-wireless | cut -d: -f1`.strip
  end
  
  puts "About to define _disconnect method"
  
  def _disconnect
    `nmcli dev disconnect #{wifi_interface}`
  end
  
  puts "Finished defining methods"
end

puts "Final methods in UbuntuModel: #{UbuntuModel.public_instance_methods(false)}"

Output:

inherited called for UbuntuModel
Methods at this point: []
Missing methods: [:_available_network_names, :_connected_network_name, :_disconnect]
About to define _available_network_names method
About to define _connected_network_name method
About to define _disconnect method
Finished defining methods
Final methods in UbuntuModel: [:_available_network_names, :_connected_network_name, :_disconnect]

The inherited hook fires before any method definitions are processed.

Alternative Approaches (And Why They Fall Short)

Method 1: `method_added` Hook

You might try using method_added to track each method as it’s defined:

class BaseModel
  UNDERSCORE_PREFIXED_METHODS = %i[_available_network_names _connected_network_name _disconnect _ip_address].freeze
  @pending_methods = []

  def self.inherited(subclass)
    subclass.instance_variable_set(:@missing_methods, UNDERSCORE_PREFIXED_METHODS.dup)
    
    subclass.define_singleton_method(:method_added) do |method_name|
      @missing_methods.delete(method_name)
      puts "Added #{method_name}, still missing: #{@missing_methods}"
    end
  end
end

This works for tracking individual methods, but it needs to run several times and doesn’t give you a clean “class is complete” callback. You’d need additional logic to determine when all methods have been defined.

Method 2: Manual Registration

You could require subclasses to manually call a verification method:

class BaseModel
  UNDERSCORE_PREFIXED_METHODS = %i[_available_network_names _connected_network_name _disconnect _ip_address].freeze

  def self.verify_underscore_methods_implemented!
    missing_methods = UNDERSCORE_PREFIXED_METHODS - public_instance_methods(false)
    unless missing_methods.empty?
      raise NotImplementedError, "#{self} must implement #{missing_methods.inspect}"
    end
  end
end

class UbuntuModel < BaseModel
  def _available_network_names; end
  def _connected_network_name; end
  def _disconnect; end
  
  verify_underscore_methods_implemented! # Manual call required
end

This works but defeats the purpose of guaranteeing that all subclasses perform verification, since it relies on the developer remembering to call the verification method in every new subclass.

The Solution: TracePoint to the Rescue

Ruby’s TracePoint API allows us to trace various execution events, including when class definitions end. We can combine this with the inherited hook to get the best of both worlds:

class BaseModel
  # Methods that subclasses must implement but are called via wrapper methods
  UNDERSCORE_PREFIXED_METHODS = %i[
    _available_network_names
    _connected_network_name
    _disconnect
    _ip_address
  ].freeze

  # Verify that a subclass implements all required underscore-prefixed methods
  def self.verify_underscore_methods_implemented(subclass)
    missing_methods = UNDERSCORE_PREFIXED_METHODS - subclass.public_instance_methods
    unless missing_methods.empty?
      raise NotImplementedError, "Subclass #{subclass.name} must implement #{missing_methods.inspect}"
    end
  end

  # Automatically verify underscore methods when a subclass is inherited
  def self.inherited(subclass)
    # Set up a trace to detect when the subclass definition ends
    trace = TracePoint.new(:end) do |tp|
      # Check if we're ending the definition of our subclass
      if tp.self == subclass
        puts "Class #{subclass} finished loading!"
        
        # Now we can safely inspect the fully-defined class
        verify_underscore_methods_implemented(subclass)
        puts "✅ All underscore methods implemented correctly!"
        
        # Clean up - disable the trace since we only need it once
        trace.disable
      end
    end
    
    # Enable the trace
    trace.enable
  end
end

class UbuntuModel < BaseModel
  def _available_network_names
    # Implementation that scans for available networks
    `nmcli -t -f SSID,SIGNAL dev wifi list`.split("\n").map(&:strip).reject(&:empty?)
  end
  
  def _connected_network_name
    # Implementation that gets current connection
    `nmcli -t -f NAME,TYPE connection show --active | grep 802-11-wireless | cut -d: -f1`.strip
  end
  
  def _disconnect
    # Implementation that disconnects from current network
    `nmcli dev disconnect #{wifi_interface}`
  end
  
  def _ip_address
    # Implementation that gets IP address
    `ip -4 addr show #{wifi_interface} | grep 'inet ' | awk '{print $2}' | cut -d'/' -f1`.strip
  end
end

class MacOsModel < BaseModel
  def _available_network_names
    # macOS-specific implementation using networksetup
    # Implementation details...
  end
  
  def _connected_network_name
    # macOS-specific implementation
    # Implementation details...
  end
  
  def _disconnect
    # macOS-specific implementation
    # Implementation details...
  end
  
  def _ip_address
    # macOS-specific implementation
    # Implementation details...
  end
end

Output when starting WiFi Wand:

Class UbuntuModel finished loading!
✅ All underscore methods implemented correctly!
Class MacOsModel finished loading!
✅ All underscore methods implemented correctly!

Perfect! Now we can inspect the fully-defined subclasses and verify they implement all required platform-specific methods.

How the TracePoint Solution Works in WiFi Wand

Inherited hook fires: When class UbuntuModel < BaseModel is encountered, inherited(UbuntuModel) is called
TracePoint setup: We create a TracePoint that listens for :end events (when end keywords are processed)
Class body processing: Ruby processes the platform-specific method implementations
End event fires: When Ruby hits the final end of the class definition, our TracePoint callback executes
Safe inspection: At this point, the OS-specific class is fully defined and we can safely verify its methods
Cleanup: We disable the TracePoint since we only need it once per class

WiFi Wand’s Method Design Philosophy

In WiFi Wand, underscore-prefixed methods follow a consistent design pattern:

Internal Implementation Methods

The underscore-prefixed methods (_available_network_names, _connected_network_name, _disconnect, _ip_address) represent the core OS-specific functionality that each platform implementation must provide.

Public API Methods

The public methods that users call (available_network_names, connected_network_name, disconnect, ip_address) are provided by the BaseModel class itself. These public methods add consistent behavior like checking if wifi is on before calling the underscore-prefixed implementations.

This separation allows:

Consistent API across all platforms regardless of OS differences
OS-specific implementations isolated to each subclass
Cross-cutting concerns handled in one place (state checking, error handling)
Clean separation between what users call and what OS’s implement

TracePoint Events the actual implementation methods that each OS must provide. They’re underscore-prefixed to indicate they’re internal implementations.

2. Public Methods (`wifi_on`, `wifi_off`, `detect_wifi_interface`, etc.)

These are the public API methods that users call. The BaseModel provides these implementations by calling the underscore-prefixed versions with appropriate conditions (like checking if wifi is on first).

This separation allows the base class to add consistent behavior (like wifi state checking) while delegating OS-specific operations to subclasses.

TracePoint Events

The :end event fires whenever Ruby processes an end keyword, which includes:

End of class definitions
End of method definitions
End of module definitions
End of blocks

That’s why we check tp.self == subclass to ensure we’re responding to the end of the specific OS-specific class we care about.

Considerations and Limitations

Performance

TracePoint has some performance overhead since it hooks into Ruby’s execution. For WiFi Wand, this isn’t a concern since:

We only create one TracePoint per OS-specific model
The trace is disabled immediately after use
Model classes are loaded once at startup

Complexity

This solution is more complex than simple hooks, but the benefit is enormous: guaranteed API consistency across all OS implementations. This is crucial for a cross-platform library like WiFi Wand where each OS model must implement the same interface.

Maintenance

The approach requires good documentation. In WiFi Wand, we clearly document the UNDERSCORE_PREFIXED_METHODS array and explain that each OS model must implement these methods.

Real-World Benefits in WiFi Wand

This TracePoint pattern has been invaluable for WiFi Wand’s development:

Early Error Detection: Missing implementations are caught immediately when the class is defined, not at runtime when a user tries to use the functionality.
Consistent API: All OS models (UbuntuModel, MacOsModel, etc.) are guaranteed to implement the same interface, making WiFi Wand’s public API consistent across platforms.
Developer Experience: When adding a new OS model, developers get immediate feedback if they forget to implement any required methods.
Self-Documenting: The UNDERSCORE_PREFIXED_METHODS array serves as clear documentation of what each subclass needs to implement.

Conclusion

Ruby’s inherited hook is powerful but has a crucial timing limitation: it fires before the subclass is fully defined. When you need to inspect or work with fully-defined subclasses in frameworks like WiFi Wand, combining inherited with TracePoint’s :end event provides an elegant solution.

This pattern gives you the automatic behavior of inheritance hooks while ensuring you have access to the complete class definition. For WiFi Wand, this means we can guarantee that every OS-specific model implements exactly the same interface, providing a consistent experience for users regardless of their operating system.

The next time you’re building a Ruby framework and find yourself frustrated that inherited doesn’t see your subclass methods, reach for TracePoint – it might be exactly what you need.

[This article was originally drafted by Claude (Anthropic) AI and subsequently edited and refined.]

Signaling Cursor AI Completion with a File Watcher and Screen Flasher

2025-05-29T00:00:00+00:00

💡 Quick Start: If you want to jump straight to implementation, see the Complete Setup Instructions at the end of this article.

Quick Access: View and Download Scripts

(Remove the '.txt' extension after downloading.)

📖 View file_watcher.py with syntax highlighting

Synchronous, Thread, and Fiber HTTP Requests in Ruby

2024-09-20T00:00:00+00:00

Introduction

In this article, we will explore different ways to make HTTP requests in Ruby and compare their performance. We will focus on three approaches: synchronous, threaded, and fiber-based. We will use a simple example of checking the availability of the same URL multiple times. Our tests will measure sending 1 to 256 requests using the different approaches.

Why do I care? I have a Ruby on Rails web site containing many links to YouTube song videos, and they are subject to being pulled by their owner or taken down due to copyright issues. I wanted to automate the checking of these links for availability as a rake task, so that I can run the check with minimal effort from time to time. The links are stored in the project in a YAML file, so it’s easy to read them into memory. However, how would I check them for accessibility?

To simplify the examples below, instead of fetching the real URL’s, I will fetch the same URL multiple times. I’ll use a URL built with "https://httpbin.org/delay/#{sleep_seconds}" to access the Internet and simulate the response delay with a sleep on the server. The examples will return an array containing the responses.

The Synchronous Approach

I started out the simple and conventional way, using Net::HTTP:

def get_responses_synchrously(count)
  logger.debug("Getting #{count} responses synchronously")
  count.times.with_object([]) do |_n, responses|
    responses << Net::HTTP.get(URI(url))
  end
end

However, the time it took was frustratingly long. How could I make this faster?

The Thread Approach

Having been a fan of threads for a long time, this was the next thing I tried. Since the number of links was just a few dozen, this number was low enough that creating a thread for each link was feasible. Note that we still use Net::HTTP.get, but each call runs in its own thread. Fortunately, Net::HTTP.get supports threaded use by yielding its thread’s control after sending the request, thereby avoiding CPU time waste while waiting for the response:

def get_responses_using_threads(count)
  logger.debug("Getting #{count} responses using threads")
  threads = Array.new(count) do
    Thread.new { Net::HTTP.get(URI(url)) }
  end
  threads.map(&:value)
end

As you can guess, this was way faster.

The Fiber Approach

I wasn’t done though – recently I finally got around to learning about Ruby fibers, and wanted to try them here. Rather than write the low level Fiber code myself, it was simpler to use @ioquatix’s (Samuel Williams’) excellent async Ruby gems (I needed async and async-http) to handle the low level plumbing. The resulting code was more complex than the previous two approaches, but not too bad (run gem install async-http if necessary):

def get_responses_using_fibers(count)
  logger.debug("Getting #{count} responses using fibers")
  responses = []
  Async do
    begin
      internet = Async::https::Internet.new(connection_limit: count)
      count.times do
        Async do
          begin
            response = internet.get(url)
            responses << response
          ensure
            response&.finish
          end
        end
      end
    ensure
      internet&.close
    end
  end.wait
  responses
end

The magic is in the Async framework and Async::https::Internet’s get method, which is fiber-aware and yields control of the CPU while waiting for a response.

Comparing the Performance Results

I did a benchmark comparing the results of fetching request counts for powers of 2 ranging from 1 to 256 (1, 2, 4, 8, 16, 32, 64, 128, and 256).

Here are the results comparing all three approaches, using the averages of several test runs:

As expected, the synchronous approach was by far the slowest, since only one request could be active at any given time. Both the thread and fiber approach were dramatically faster, and not that different from each other. To zoom in on the difference between the thread and fiber approach, this graph omits the synchronous approach:

What do we make of this though? Which should we choose to use?

Fibers vs. Threads

If one knows that the numbers will always fall within the bounds of 1 to 256, then it probably doesn’t much matter which to use. However, if there is a possibility of higher request counts, then fibers make more sense. Here are some ways in which threads and fibers differ:

Threads:

OS-Mapped: In Ruby versions 1.9 and later, Ruby threads are mapped to operating system threads. Each thread has its own dedicated stack and other resources allocated by the OS.
Context Switching Overhead: Switching between threads involves saving and restoring the entire execution context (registers, stack pointers, etc.), which is a relatively expensive operation.
System Limits: The operating system typically imposes limits on the number of threads a process can create due to resource constraints.

Fibers:

User-Level: Fibers are a user-level construct managed entirely within the Ruby interpreter. They share the same stack and other resources with the thread they’re running in.
Cooperative Scheduling: Fibers explicitly yield control to each other, making context switching much faster and less resource-intensive.
Lightweight: Due to their cooperative nature and shared resources, fibers have a much smaller memory footprint than threads.

Operating System Open File Handle Limits

A file handle (aka “file descriptor”) is an object (typically a small non-negative integer) stored in a variable on the OS level used to refer to a resource, which can be a file, pipe, socket, or terminal. The OS restricts the number of file handles that can be used by a given process. In our testing, we need to stay within or increase that limit.

In general, the synchronous approach results in only one file handle being used at a time for all the requests. In contrast, the thread and fiber approaches may theoretically use file handles for all the requests at the same time, since they do not wait for one to finish to start another.

Even with as few as 256 simultaneous requests, the operating system session’s file handle limit may be exceeded. If you get an error saying that all the process’ file handles have been used, in Linux and Mac OS you can use ulimit to increase the maximum file handle count (used for both files and network sockets) for the terminal session, and then rerun the program. For example: ulimit -n 2048 && my-program. However, ulimit will only do this successfully if the systemwide maximum file count is large enough to accommodate it.

Threads are far more heavyweight than fibers, so for large request counts, one would need to implement some kind of thread pooling, and this would probably result in far fewer requests per second than fibers. Sam Williams posted a YouTube video (RubyConf Taiwan 2019 - The Journey to One Million by Samuel Williams - YouTube) in which he showed one million fibers running network requests!

JRuby

No discussion of Ruby concurrency is complete without a reminder that even with multiple threads, C Ruby’s Global Interpreter Lock (aka “the GIL”) guarantees that only one CPU can be used at a time. In contrast, JRuby (Ruby running on the Java Virtual Machine), threads do run truly concurrently, on multiple CPU’s. This can make threading in JRuby much more performant.

That said, these requests are not making heavy use of the CPU, and JRuby threads (really, Java threads) are still far more heavyweight than fibers, so even with JRuby fibers may be the better choice.

Large Request Counts

This article covered small numbers of requests, but what if you need to make thousands or millions of requests? In that case,

Thread Pooling: If you are using threads, you may need to implement a thread pool to limit the number of threads created and manage the requests.
Fiber Pooling: If you are using fibers, you may need to implement a fiber pool to limit the number of fibers created and manage the requests.
Async Gems: If you are using fibers, you may want to consider using the async gems to handle the low-level plumbing for you. This will make your code simpler and more maintainable, but will add a dependency to your project.
Operating System Limits: Be aware of the operating system’s file handle limits and ensure that you stay within them.
JRuby: If you are using JRuby, you may be able to take advantage of true concurrency with threads.
Performance Testing: Thoroughly test your code with the expected number of requests to ensure that it performs as expected.
Error Handling: Ensure that your code handles errors gracefully and does not crash when an error occurs.
Logging: Add logging to your code to help you diagnose issues and monitor performance.
Code Complexity: Consider the complexity of your code and choose the approach that is simplest and easiest to maintain.
Code Review: Have your code reviewed by a colleague to ensure that it is correct and follows best practices.
Documentation: Document your code to make it easier for others to understand and maintain.
Testing: Write tests for your code to ensure that it works as expected and to catch any regressions.
Performance Monitoring: Monitor the performance of your code in production to identify any bottlenecks and optimize as needed.

Conclusion

Which approach to use depends on a number of factors:

What will be the average request count?
What will be the maximum request count?
How often will this be used?
How important is faster completion?
Do I want or need to avoid the additional dependency of the async gems?
How important is code simplicity?

Thorough research may be necessary to determine the very best approach for any given situation, but here is one policy that balances performance and simplicity:

Request Count	Approach
n <= 3	Synchronous
4 <= n <= 16	Threaded
n > 16	Fiber

Addendum

The complete Ruby program used to measure request performance can be found at https://gist.github.com/keithrbennett/719af73894458a4378aa3e3a5cc9b70a and is also pasted here:

#!/usr/bin/env ruby
# frozen_string_literal: true

# IMPORTANT: You may need to increase the number of available file handles available to this process.
# The number should be greater than the maximum number of requests you want to make, because each process
# opens at least 3 file handles (stdin, stdout, stderr), and other files may be opened during the program's
# run (e.g. for the logger).
# ulimit -n 300 && scripts/compare_request_methods.rb

require 'async/http/internet' # gem install async-http if necessary
require 'awesome_print'
require 'benchmark'
require 'json'
require 'logger'
require 'net/http'
require 'pry'
require 'yaml'

# These are the external gems that must be installed for the program to run.
REQUIRED_EXTERNAL_GEMS = %w[async-http awesome_print pry].freeze

Thread.abort_on_exception = true
Thread.report_on_exception = true

class Benchmarker
  attr_reader :logger, :request_count_per_run, :sleep_seconds, :url

  def initialize(request_count_per_run, sleep_seconds, logger)
    @request_count_per_run = request_count_per_run
    @sleep_seconds = sleep_seconds
    @logger = logger
    @url = "https://httpbin.org/delay/#{sleep_seconds}"
  end

  def get_responses_synchrously(count)
    logger.debug("Getting #{count} responses synchronously")
    count.times.with_object([]) do |_n, responses|
      responses << Net::HTTP.get(URI(url))
    end
  end

  def get_responses_using_threads(count)
    logger.debug("Getting #{count} responses using threads")
    threads = Array.new(count) do
      Thread.new { Net::HTTP.get(URI(url)) }
    end
    threads.map(&:value)
  end

  def get_responses_using_fibers(count)
    logger.debug("Getting #{count} responses using fibers")
    responses = []
    Async do
      begin
        internet = Async::https::Internet.new(connection_limit: count)
        count.times do
          Async do
            begin
              response = internet.get(url)
              responses << response
            ensure
              response&.finish
            end
          end
        end
      ensure
        internet&.close
      end
    end.wait
    responses
  end

  def self.call(request_count_per_run, sleep_seconds, logger)
    self.new(request_count_per_run, sleep_seconds, logger).call
  end

  def output_results(results)
    logger.info('-' * 60)
    logger.info(results.to_json)
    ap(results)
  end

  def call
    logger.info("Starting run with #{request_count_per_run} requests each sleeping #{sleep_seconds} seconds")
    results = {
      time:        Time.new.utc,
      sleep:       sleep_seconds,
      count:       request_count_per_run,
      fibers:      Benchmark.measure { get_responses_using_fibers(request_count_per_run) }.real,
      threads:     Benchmark.measure { get_responses_using_threads(request_count_per_run) }.real,
      synchronous: Benchmark.measure { get_responses_synchrously(request_count_per_run) }.real,
    }
    output_results(results)
    results
  end
end

class Runner
  def self.call() = new.call

  def setup_logger
    logger = Logger.new('compare_request_methods.log')
    logger.level = Logger::INFO

    logger.info('=' * 60)
    logger
  end

  # Measure the time it takes to run a block of code
  # @return [Array] The return value of the block and the duration in seconds
  def time_it
    start_time = Process.clock_gettime(Process::CLOCK_MONOTONIC)
    return_value = yield
    end_time = Process.clock_gettime(Process::CLOCK_MONOTONIC)
    duration_in_seconds = end_time - start_time
    [return_value, duration_in_seconds]
  end

  def write_results(logger, results, duration_secs)
    timestamp = Time.now.utc.strftime('%Y-%m-%d-%H-%M-%S')
    File.write("#{timestamp}-results.yaml", results.to_yaml)
    logger.info(results.to_json)
    puts("Done. Entire suite took #{duration_secs.round(2)} seconds.")
  end

  def call
    counts = [1, 2, 4, 8, 16, 32, 64, 128, 256]
    logger = setup_logger
    puts "Starting run with counts: #{counts.join(', ')}"

    results, duration_secs = time_it do
      counts.map { |count| Benchmarker.call(count, 0.0001, logger) }
    end

    write_results(logger, results, duration_secs)
  end
end

class GemChecker
  def self.call(required_external_gems) = new.ensure_gems_available(required_external_gems)

  def gem_exists?(gem_name)
    begin
      gem(gem_name)
      true
    rescue Gem::MissingSpecError
      false
    end
  end

  def find_missing_gems(required_gems)
    required_gems.reject { |name| gem_exists?(name) }
  end

  def ensure_gems_available(required_external_gems)
    missing_gems = find_missing_gems(required_external_gems)
    if missing_gems.any?
      puts "Need to install missing gems: #{missing_gems.join(', ')}"
      exit(-1)
    end
  end
end

GemChecker.call(REQUIRED_EXTERNAL_GEMS)
Runner.call

AutoPilot, CoPilot, or AI Assistant?

2024-01-30T00:00:00+00:00

AI Downsides

In New GitHub Copilot Research Finds ‘Downward Pressure on Code Quality’, Visual Studio magazine cites a GitClear study that points out that AI for coding results in less maintainable (and therefore costlier) code.

They point out, for example, that copying and pasting code in multiple locations where it is needed, rather than moving it to a single location and referring to it there, is prone to producing repetitive code, that is, code that is not DRY.

In addition, at the time of this writing (January 2024), AI still makes plenty of mistakes; that is, a) misunderstandings of intent, b) hallucinations, and c) downright errors.

Resisting Oversimplification

While the study describes in great detail the pitfalls it found in AI-assisted coding, we must not succumb to the oversimplification so prevalent in these times and generalize that “AI is 100% bad”. We need to realize that the positive or negative value of AI depends on this crucial nuance: the extent to which the developer relies on and defers to AI, rather than supervising it with a critical eye.

The name “Copilot” is apt. It must not be “Autopilot”. However, even “Copilot” may not sufficiently express the subservient position that AI should take; “AI Assistant” would probably be more accurate, and is exactly the name used by JetBrains, the company that produces outstanding IDE’s.

The Optimum Balance

An organization that mandates excessive reliance on AI to quickly reduce labor costs will have a rude awakening when, over time, the poor code quality reveals itself in the form of reduced quality and increased cost.

On the other hand, an organization that completely ignores the immense productivity boost that coding AI offers will “leave money on the table” and fall behind the competition.

Organizational Responsibility

To quote my favorite paragraph in the study:

“If there isn’t a CTO or VP of Engineering who actively schedules time to reduce tech debt, you can add “executive-driven time pressures” to the list of reasons that newly added copy/paste code will never be consolidated into the component libraries that underpin long-term development velocity.”

In other words, resisting the temptation to build working code quickly in the short term at the expense of the medium and long term requires will power and effort, plus initiative or at minimum support from the highest levels of the organization. Those organizations that are not able or willing to delay gratification will suffer the consequences.

Investing in code quality pays off well, but does have an initial cost. In the case of AI, this will be especially important.

Conclusion

The bottom line is that we still need expert software developers to know how to use the tool, appreciating its knowledge and assistance, but taking the lead to verify and integrate it correctly. The AI landscape is changing at astonishing speed. Organizations and their developers need to reevaluate tools and policies frequently to ensure they are making optimal use of this game changing technology.

Installing International Language Keyboard Input Methods on Linux Mint 20 Cinnamon & Mate

2020-08-23T00:00:00+00:00

I just started using a desktop system using Linux Mint 20 Cinnamon, and wanted to make sure that keyboard input methods worked for Thai and Korean. This took me down a long road, consulting several blog articles, questions, and answers, so I decided to document it to save others (and future me!) this effort.

I’ve also briefly tested that this procedure works with Arabic, French, Hebrew, Russian, and Swedish, so I’m fairly confident that it will work for any supported language and input method.

Below are the instructions for Linux Mint Cinnamon; I believe they will be identical for Mate.

Language Support

Press the {Super} key, start typing Languages, and select the “Languages” application.
Click “Install/Remove Languages” at the bottom right.
If requested, enter your login password to grant superuser rights to “Languages”.

For each language you want to add:

Click “Add”
Start typing the name of the language, select the language you want, and then click the “Install” button. There may not be any confirmation that it has completed.
Find the newly installed language in the list (again, you can start to type it to find it), select it, and click the “Install Language Packs” button.
Confirm any confirmation dialogs as necessary.

If…

The remainder of work to be done involves the IBus input method framework.

If at any point during this procedure, you see a dialog saying “The IBus daemon is not running. Do you wish to start it?”, click “Yes”.

Also, if you encounter a dialog saying “IBus has been started! If you cannot use IBus, add the following lines…”, just click OK. I have seen this happen but have never needed the information displayed in the dialog.

Install IBus

Install the IBus keyboard input management framework and the Hangul (native Korean alphabet) support:

sudo apt install -y ibus-m17n ibus-hangul

Add IBus to the Startup Applications List

Press the {Super} (Windows/Mac) key, type startup and select “Startup Applications”.

Click the “+” button.
Select “Custom command”.
For Name: IBus
For Command: ibus-daemon
Click “Add”.

Configure IBus

Press the {Super} key and type ibus. Select “IBus Preferences”.

Add a keyboard shortcut

Cinnamon uses {Super}{Space} to navigate panel entries, and I have not found a way to disable that, so I assign another shortcut for language switching, {Ctrl}{Super}{Alt}-k (“k” for keyboard). Here’s how:

On the “General” tab:

click the button labelled “…” to the right of the “Next Input Method” text field
with “{Super}" selected, click the "Delete" button
Replace any text in the “Key code” input field with “k”
check “Control”, “Alt”, and “Super”
click the “Add” button
click the “OK” button

You may not need to do this if you are using Mate and the {Super}{Space} key combination is available for IBus to use.

Also, I found that {Ctrl}{Super}{Alt}-k did not work when pressed while certain input methods were active. I briefly researched how to disable Cinnamon’s use of {Super}{Space} but could not find an answer. Anyone? Assigning multiple key combinations to this action is supported, so that is another approach.

Add input methods

On the “Input Method” tab, click “Add”.

For each desired input method, click your desired language, or if it is not listed (as with Korean and Thai), click the three vertical dots entry, click the text field to give it focus, and type the language into the text field, then select the entry below the language name that you want. In the case of Korean, there is only one (“Hangul”). Click “Add”.

Reboot and Test

Reboot the system. (If you want to save time, you could log out and then in again instead of rebooting, but that will not verify that IBus was started on system startup.)

Test switching input methods with {Ctrl}{Super}{Alt}-k, and typing text into an application that can accept it.

The Language Panel Applet

There are two methods for invoking input method selection:

the keyboard shortcut ({Ctrl}{Super}{Alt}-k, {Super}{Space}, etc.)
the panel applet

The panel applet is on the system panel, and will display the currently selected language. Clicking it will display a list of languages from which you can select a different one. This applet can be used as a fallback mechanism for changing language if the keyboard shortcut does not work.

Unfortunately, the text on this indicator is displayed in a dark blue text that is difficult to see against the black background, so it may take some effort to find. The Korean language setting deals with this by showing a colorful icon instead of text.

Cautions

Invoking Input Method Selection

As mentioned, if the keyboard shortcut for selecting an input method does not work, remember the panel applet and use that instead.

Know Your Input Methods

The Korean input method uses a {Shift}{Space} toggle key combination to toggle between Hangul and English character input. When you switch into Korean input mode from another language, it will initially be in English mode, so it may not be obvious that you have successfully switched to Korean. Press {Shift}{Space} to toggle to Hangul input mode.
If you’re new to a language, there may be things to learn about that language’s computer input methods that are not taught in school or otherwise obvious. If the input does not appear to work correctly, make sure you’re not overlooking a feature of that input method.
Some languages have many input methods from which to choose, and some may be much better than others for your purposes. For example, some are Dvorak layouts of interest only to the hardiest of souls.

Language-Specific Configuration

There may be language specific requirements with other languages like the need to install the ibus-hangul package for Korean.

You’re done!

Thanks for reading, and let me know in a comment on the dev.to article page if you have any corrections or improvements to offer.

Add Bootstrap to Rails 6 with Two Shell Commands

2020-05-07T00:00:00+00:00

Introduction

In this article I will discuss a very simple approach to configuring your Rails 6 application to work with Bootstrap. An entire Rails repo is provided, including commit history, but this might be all you need:

curl https://raw.githubusercontent.com/keithrbennett/rails-bootstrap-example/master/0001-Add-Bootstrap-configuration.patch | git apply -
yarn add bootstrap jquery popper.js

I recently wanted to add Bootstrap to a new Rails 6 application, but even after reading the documentation and several blog articles, success eluded me.

What finally worked was to follow along with this article by Chris Oliver – well, more precicely, the video linked to in the article. In it, he shows exactly what to do – and doing what he said to do worked for me.

Even Simpler – A Patch

In order to even further simplify the process for future developers, I generated a patch file to make the changes needed to provide the correct configuration. (These changes do assume a new Rails application, so it’s possible that some modification to the changes would be required for existing projects.)

You can make the changes to your existing project or a fresh one generated with rails new. Here’s how to do it:

Change directory to your project root.

I suggest you have a “clean working tree” (no git-relevant changes since the most recent commit) before you apply the patch; this will make it easier to revert the change or limit your next commit to only the Bootstrap configuration change.

You can download the patch in its raw format (i.e. only the unadorned text and not the HTML web page displaying it) to your local filesystem with the following command:

curl -o bootstrap.patch https://raw.githubusercontent.com/keithrbennett/rails-bootstrap-example/master/0001-Add-Bootstrap-configuration.patch

Then apply the patch:

git apply bootstrap.patch

Alternatively, you can combine the above two steps into one:

curl https://raw.githubusercontent.com/keithrbennett/rails-bootstrap-example/master/0001-Add-Bootstrap-configuration.patch | git apply -

You can see the changes with a git diff.

Adding the JavaScript Libraries

At some point (either before or after the patch) you will need to add the necessary JavaScript libraries:

yarn add bootstrap jquery popper.js

This will make changes to package.json and yarn.lock that will need to be committed.

Testing that Bootstrap Works

We’ll want to exercise Bootstrap to verify that it is working correctly.

The index.html.erb file in the repo uses Bootstrap colored border spinners requiring both CSS and JavaScript provided by Bootstrap, and is a good test. Of course, you can find many other components to use in the Bootstrap docs.

The rest of this section discusses setting up a sample app using the patches provided. Here is a list of all the patches:

# already used above, to configure Bootstrap
0001-Add-Bootstrap-configuration.patch                       

# These next 3 can be used to put something in the
# home page to test that Bootstrap is working:
0002-rails-g-controller-home-index.patch
0003-Change-root-route-to-go-to-new-page.patch
0004-Add-Bootstrap-color-border-spinners-to-page.patch

These patches are in the project root of the sample repo.

Applying patches #2 through #4 will set up the sample code to show that Bootstrap is working. Here are commands that will apply the patches directly from Github (without creating patch files on your local filesystem):

curl https://raw.githubusercontent.com/keithrbennett/rails-bootstrap-example/master/0002-rails-g-controller-home-index.patch | git apply -
curl https://raw.githubusercontent.com/keithrbennett/rails-bootstrap-example/master/0003-Change-root-route-to-go-to-new-page.patch | git apply -
curl https://raw.githubusercontent.com/keithrbennett/rails-bootstrap-example/master/0004-Add-Bootstrap-color-border-spinners-to-page.patch | git apply -

This is all you should need to do! At this point you can run rails s and connect to it in your browser. If all goes well you will see something like this:

Git’s Patch Support

You may have noticed that the patch file names were numbered and contained the first part of the commit messages in the names. This was not something I did myself, this was done automatically by git. Git has great patch support, and all I needed to do to generate the patches was issue this command:

$ git format-patch HEAD~4
0001-rails-g-controller-home-index.patch
0002-Change-root-route-to-go-to-new-page.patch
0003-Add-Bootstrap-color-border-spinners-to-page.patch
0004-Add-patch-files-git-format-patch-HEAD-4.patch

The HEAD~4 told git how far back I wanted to start (4 commits).

As you saw above, applying a patch to a code base is as simple as:

git apply something.patch

Conclusion

I hope that the time I invested in creating and documenting this simplifying procedure will pay off in the form of time saved for you. If you think I could be useful on your project, or would just like to say hello, please give me a holler at kbennett@bbs-software.com.

Note: The patch (#1) originally published with this article did not include some configuration code needed to expose JQuery and Popper, which could have been a problem implementing customizations. It was fixed on 2020-05-12. You can see the changes here.

The Case for Stabby Lambda Notation

2019-11-30T00:00:00+00:00

The Stabby Lambda (`->`)

Although the -> “stabby lambda” notation has been available for creating lambdas since Ruby version 1.9, old habits die hard and acceptance and adoption has been slow. In this article I will explain why I recommend using it instead of the lambda notation.

Stabby Notation as an Indicator of Preferred and Default Proc Type

In a previous article, “lambdas Are Better Than procs”, I proposed that lambdas should be used rather than procs in almost all cases, given that they are safer in terms of argument count checking and return behavior.

So it makes sense that -> should create a lambda and not a proc. (As an aside, it always puzzles me when people use the term stabby proc, when it creates a lambda.)

One way to look at it is, by using the stabby lambda notation, we are saying “make me Ruby’s implementation of an objectless function”. This is at a level higher than “make me a lambda” or “make me a proc”, and is probably a better interface to the programmer, especially the newer Rubyist.

`->`’s Picture-Like Notation

The picture-like notation -> is quite different from the lambda and proc forms, because although all result in method calls that create Proc instances, lambda and proc look like method calls, while -> does not, instead appearing more like a language construct. On the higher level, it really is a language construct, and the fact that a method needs to be called to create a lambda is an implementation detail that should not matter to the programmer.

The striking appearance of -> says to the reader “take note, something different is happening here, this marks the beginning of a definition of executable code that will probably be called somewhere else”. If a picture is worth a thousand words, then a text picture like -> is worth, well, at least ten.

The Need for Visual Differentiation

Unlike other code in a method, a lambda’s code is not called in sequence (unless it is immediately called as a self invoking anonymous function, but this is rare). Also, sometimes a lambda can be used as if it were a nested method, containing lower level code that may be called multiple times in the method in which it was defined. For these reasons, a pictorial indication setting it apart from other code in the method is especially helpful.

Rubocop

Rubocop is a very useful tool for normalizing code style. For better or worse though, Rubocop’s defaults constitute implicit recommendations, and deviating from the defaults can require lengthy and contentious team discussions. Because of this potentially high cost of overriding the defaults, it is important that the basis in reasoning for the selection of the default be sound.

Rubocop’s default setting for lambdas is to use -> with lambda one-liners but lambda for multiline lambdas. While this is not a matter of monumental importance, I believe it’s misguided and should be changed.

My guess is that it is intended to mirror the Ruby code block notation convention of {..} for single line blocks and do...end for multi-line blocks. However, the code block case is different because the do and end are at the end and beginning of the line, respectively (though it is true that if there are arguments they will appear after the do). Although the indentation of the code block within the lambda do...end makes it easy to see that something is going on, it is easy to miss the lambda and assume it is a normal code block. The pictorial nature of -> reduces this risk.

I believe that the Rubocop default should be changed to prefer (or at minimum permit) -> in all cases.

Note: Since writing this article I posted an issue on the Rubocop project site here.

Conclusion

Lambdas are, thankfully, first class objects in Ruby. That is, they can be passed to and returned from methods, and can be assigned to variables. This is a pretty major construct, and I believe a special notation (->), rather than a method name (lambda) is justified and helpful. While it is true that class, module, and def also mark the beginning of major language constructs, they are likely to be the first token on a line, whereas lambdas are usually assigned to variables or passed to methods or other lambdas, and are not.

The conciseness and pictorial nature of -> encourage the use of lambdas, and in my opinion, that is a Good Thing. Lambdas are underused in the Ruby community, and many opportunities for cleaner and clearer code are missed.

lambdas Are Better Than procs

2019-11-19T00:00:00+00:00

Many Rubyists believe that lambda and nonlambda Procs are pretty much the same and that choosing which one to use is a subjective preference. This is an unfortunate fallacy.

This article will attempt to achieve two purposes:

1) to explain the difference between lambdas and procs

2) to persuade you to use lambdas unless there is a compelling reason not to

There are many resources available that explain lambdas and procs ^[1], and I will assume you know at least a little about them.

Before we look at some examples, here are some characteristics of lambdas and procs:

a Proc instance can be either lambda or a proc ^[2]
all lambdas are Procs
all procs are Procs
code blocks behave like procs
you can determine the kind of Proc by calling lambda? on it

Arity (Argument Count) Checking Behavior Differences

A lambda, like a method, strictly enforces its argument count, but a proc does not. When we call a proc with the wrong number of arguments, there are no complaints by the Ruby runtime ^[3]:

2.6.5 :006 > pfn = proc { |arg| }
 => #
2.6.5 :007 > pfn.call
 => nil

In contrast, when we do the same with a lambda, we get an error ^[4]:

2.6.5 :002 > lfn = ->(arg) {}
    => #
   2.6.5 :003 > lfn.call
   ...
   ArgumentError (wrong number of arguments (given 0, expected 1))

Which behavior would you prefer?

Clearly, arity checking is helpful, and we abandon it at our peril.

Return Behavior Differences

What happens when you pass a code block somewhere, and it executes a return? Does it return from the block? Well, yes, but it does much more than that; it returns from the method that yielded to the block. procs behave the same way; in addition to returning from themselves, they will return from the method in which they were called:

def using_proc
  pfn = proc { return }
  puts "Before calling"
  pfn.call
  puts "After calling"
end

# ...
2.6.5 :015 > using_proc
Before calling

Before proceeding to the lambda behavior, I’d like to point out that this proc behavior is such that implicit and explicit returns do very different things. An implicit return will return from the proc, but an explicit return will return from the context that called it! Weird, eh? Here is the same code, but without the explicit return; the proc will end and exit naturally:

def using_proc_without_return
  pfn = proc { }
  puts "Before calling"
  pfn.call
  puts "After calling"
end
# ...
2.6.5 :007 > using_proc_without_return
Before calling
After calling

When we first learn Ruby, we learn that a return at the end of a method is redundant (it is, of course), but in the case of the proc (and code block) it is not!

In contrast, a lambda’s return returns from itself to the context that called it:

def using_lambda
  lfn = -> { return }
  puts "Before calling"
  lfn.call
  puts "After calling"
end
# ...
2.6.5 :008 > using_lambda
Before calling
After calling

A `lambda` is More Method-Like Than a `proc`

In both of the above cases, the lambda behaves more like a method than a proc does. The newer ->(args) notation for creating a lambda reveals that intent by defining the arguments as a method does, in a parenthesized list, and is therefore preferable to the older lambda notation:

New:

fn = ->(arg1, arg2) { ... }

Old:

fn = lambda { |arg1, arg2| ... }

Conclusion

Here are some principles I’ve learned to code by:

prefer simplicity to complexity
limit things to the narrowest possible scope
specify things with minimal ambiguity
use language features that minimize the risk of errors

Regarding everything said so far, the lambda wins over the proc. There is no reason to use a proc unless you specifically need the odd and potentially hazardous behaviors described above.

You may think it will never matter in your case. Maybe you’re never calling the lambda yourself, but passing it to a framework such as Rails that is doing all the calling. Nevertheless, if given the added protection for free, why would you not want it? Especially since the -> notation is somewhat pictorial and more concise?

Footnotes

[1] There are many good resources; here are some that I have produced (articles and a conference talk):

Using Lambdas to Simplify Varying Behaviors in Your Code
Ruby Enumerables Make Your Code Short and Sweet
Functional Programming in Ruby, video of a talk given at Functional Conf in Bangalore, India in 2014 ↩

[2] This terminology is unfortunate, as Proc and proc, when spoken, sound identical. ↩

[3] In this article I’ve used the .call variant of calling a Proc because it is the most obvious for the reader, but in practice I prefer the shorthand notation .(). ↩

[4] The shorthand -> can be used in place of the lambda keyword to more succinctly define a lambda. ↩

Ruby Enumerables Make Your Code Short and Sweet

2019-11-13T00:00:00+00:00

One of the most amazing things about Ruby is the richness of its Enumerable library; there are so many things it can do. Another is Ruby’s ability to express intent with the utmost conciseness and clarity. However, out in the wild I very often see code that fails to take full advantage of these qualities.

As a contrived example, let’s say we’re keeping track of letter frequencies in a document. We define a class to contain them as:

LetterFrequency = Struct.new(:letter, :frequency, :vowel?)

I’ve seen a lot code that looks like this:

def filtered_and_transformed_records_1(records)
  results = []

  records.each do |record|
    next unless record.vowel?
    results << [record.letter, record.frequency]
  end

  results
end

In more primitive languages one must use these approaches, but in Ruby we have some major refactorings that can make this code much, much simpler.

First, we can use each_with_object to eliminate the need for the explicit initialization of the function-local variable containing the array and its explicit return, on the first and last lines of the method:

def filtered_and_transformed_records_2(records)
  records.each_with_object([]) do |record, results|
    next unless record.vowel?
    results << [record.letter, record.frequency]
  end
end

I say function-local because we do need the block-local variable results inside the each_with_object block. However, we’ve narrowed the scope of the results variable, and that’s always a good thing.

each_with_object is like each except that it will pass two variables to the block instead of one. In addition to the object from the Enumerable that each passes, it passes the object you are using to accumulate results. You initialize the accumulator by passing its initial value to the each_with_object method. In this case we are passing a newly created empty array.

each_with_object’s return value is the accumulator object, so you don’t need to specify the accumulator explicitly for it to be the value returned by the method.

The each_with_object usage may not feel natural at first, but once you’ve seen it a few times your mind will parse it with almost zero effort. (By the way, I always had trouble remembering the order of its arguments until I realized that they were in the same order as in the method name itself; each for the enumerated object and object for the accumulator object.)

The second refactoring is instead of using control flow constructs like next, we can use the Enumerable methods select or reject. We could refactor the code further into:

def filtered_and_transformed_records_3(records)
  records.select(&:vowel?).each_with_object([]) do |record, results|
    results << [record.letter, record.frequency]
  end
end

After this refactoring, we see the filter where it is more appropriate and helpful. Instead of it being on a line inside the block, it’s just a few characters immediately after the input array (records.select...).

We’ve already simplified this method quite a bit, but there’s even more we can do. Because select returns the filtered array, we can simplify even further by using map instead of each_with_object!:

def filtered_and_transformed_records_4(records)
  records.select(&:vowel?).map { |record| [record.letter, record.frequency] }
end

Although as software developers our mission is to deliver functionality, the other side of that coin is to do so as simply as possible. Put otherwise, we need to remove accidental complexity (a.k.a. incidental complexity) so that only the essential complexity remains. The functional approaches described here are extremely effective at doing this. We’ve ended up with a simple one-liner.

Whenever you start feeling that your code is getting verbose or awkward, ask yourself “could I improve this code with Enumerable?” The answer may well be yes.

For your reference, here is a file that contains the methods in the article, and verifies that they all produce the same result.

Automate the Little Things Too

2019-09-24T00:00:00+00:00

We developers automate highly complex tasks, but when it comes to the smaller repetitive tasks, we tend to do things manually, or fail to do them at all. By combining Ruby with robust and richly functional command line tools such as MPlayer, we can save ourselves lots of time and have fun in the process.

I recently decided that it would be nice to trim my collection of many video files downloaded from my phones over the years. Realizing this would be quite tedious, I asked myself “would this be easier with Ruby?” The answer, of course, was Yes!

Integrating MPlayer and Ruby

MPlayer is a Unix command line multimedia player that can be installed with your favorite package manager (e.g. brew, apt, or yum). By driving MPlayer from Ruby, we can create a workflow that will enable you to view and decide about video files with a minimum of keystrokes, without needing to use the mouse.

Files to process are specified on the command line. Multiple arguments can be specified, either absolute or relative, and either with or without wildcards. All filespecs are normalized to their absolute form so that duplicates can be eliminated.

MPlayer plays each file for the user, responding to cursor keys to move forward and backward in time, change the speed, etc. I recommend viewing the man page (man mplayer), but here are the most relevant options:

keyboard control
      LEFT and RIGHT
           Seek backward/forward 10 seconds.
      UP and DOWN
           Seek forward/backward 1 minute.
      PGUP and PGDWN
           Seek forward/backward 10 minutes.
      [ and ]
           Decrease/increase current playback speed by 10%.
      { and }
           Halve/double current playback speed.
      BACKSPACE
           Reset playback speed to normal.

When the user has seen enough to make a decision, q or [ESC] can be pressed, and MPlayer returns control to the Ruby script, which accepts a one character response to mark it to be saved (s), deleted (d), or marked as undecided (u) for future reprocessing; or q to quit the application.

The High Level View

Here is the highest level method in the script:

def main
  check_presence_of_mplayer
  create_dirs
  puts greeting
  files_to_process.each do |filespec|
    play_file(filespec)
    print disposition_prompt(filespec)
    destination_subdir = get_disposition_from_user
    `mv #{filespec} #{destination_subdir}`
    log(filespec, destination_subdir)
  end
end

Using Subdirectories

For simplicity of implementation and added safety, the application “marks” each multimedia file by moving it to one of the three subdirectories it has created, based on the user’s choice. The user selects d for deletes, s for saves, or u for undecideds. create_dirs creates the three subdirectories:

def create_dirs
  %w{deletes  saves  undecideds}.each { |dir| FileUtils.mkdir_p(dir) }
end

When the user is finished processing all files, they will probably want to move any files that have been moved to ./undecided back to . and run the program again.

Finally, when there are no files left in undecided, one will probably want to do something like this:

rmdir undecideds
rm -rf deletes
mv saves/* .
rmdir saves

An Example

For example, let’s say you run the following command:

organize-av-files 'video/*mp4' 'audio/*mp3'

MPlayer will begin playing the first file. When you are ready to finish viewing it, you will press q or ESC, and be presented with a prompt like this:

/Users/kbennett/android/video/20160102_234426.mp4:
s = save, d = delete, u = undecided, q = quit:

Type your response choice and then [Enter]. The program will move the file as appropriate, and immediately start playing the next file.

Shell vs. Ruby Wildcard Expansion

Be careful when using wildcards. If you enter *mp4 in a directory with 10,000 MP4 files, the shell will try to expand it into 10,000 arguments, which might exceed the maximum command line size and result in an error. You can instead quote the filemask (as '*mp4'), and it will then be passed to Ruby as a single argument, and Ruby will perform the expansion. You can usually use double quotes, but be aware that the single and double quotes behavior differs (see this helpful StackOverflow article).

One case where the shell’s expansion would be preferable is with the use of environment variables in the filespec ($FOO is more concise than ENV['FOO']), and in the case of using ~ for users other than the current user (e.g. ~someoneelse).

Also…

This workflow can be used with any multimedia files recognized by MPlayer, and that includes audio files.
There are many, many nice-to-have features that have not been implemented, since speed of implementation was a high priority. Feel free to add your own!
Although using Ruby probably enables writing the most concise and intention-revealing code, other languages such as Python would do fine as well.
The code for this script (“organize-av-files”) is currently at https://gist.github.com/keithrbennett/4d9953e66ea35e2c52abae52650ebb1b.

Conclusion

I hope you can see that with a modest amount of code you can build a highly useful (albeit not fancy) automation tool. The amount of expected use and the benefit per use determines the optimum amount of effort, and you have the freedom to choose any point in that continuum. The notion that all applications need to be feature-rich is not a useful one, and often results in inaction altogether.

Ruby is a great tool for this sort of thing. Why not use it?

— The End —

For your convenience, the script is displayed below:

#!/usr/bin/env ruby

# organize-av-files - Organizes files playable by mplayer
# into 'saves', 'deletes', and 'undecideds' subdirectories
# of the current working directory.
#
# Be careful, if you specify files to process in multiple directories,
# they will all be moved to the same subdirectories, so they will no
# longer be organized by directory, and if there are multiple files
# of the same name, some may be lost if overwritten.
#
# stored at:
# https://gist.github.com/keithrbennett/4d9953e66ea35e2c52abae52650ebb1b


require 'date'
require 'fileutils'
require 'set'

LOG_FILESPEC = 'organize-av-files.log'

def create_dirs
  %w{deletes  saves  undecideds}.each { |dir| FileUtils.mkdir_p(dir) }
end


def check_presence_of_mplayer
  if `which mplayer`.chomp.size == 0
    raise "mplayer not detected. "
        "Please install it (with apt, brew, yum, etc.)"
  end
end


# Takes all ARGV elements, expands any wildcards,
# converts to normalized (absolute) form,
# and eliminates duplicates.
def files_to_process

  # Dir[] does not understand ~, need to process it ourselves.
  # This does *not* handle the `~username` form.
  replace_tilde_if_needed = ->(filespec) do
    filespec.start_with?('~/')                    \
        ? File.join(ENV['HOME'], filespec[2, -1]) \
        : filespec
  end

  # When Dir[] gets a directory it returns no files.
  # Need to add '/*' to it.
  add_star_to_dirspec_if_needed = ->(filespec) do
    File.directory?(filespec)      \
        ? File.join(filespec, '*') \
        : filespec
  end

  # Default to all nonhidden files in current directory
  # but not its subdirectories.
  ARGV[0] ||= '*'

  all_filespecs = ARGV.each_with_object(Set.new) do |filemask, all_filespecs|
    filemask = replace_tilde_if_needed.(filemask)
    filemask = add_star_to_dirspec_if_needed.(filemask)

    Dir[filemask]                          \
        .map { |f| File.absolute_path(f) } \
        .select { |f| File.file?(f) }      \
        .each do |filespec|
      all_filespecs << filespec
    end
  end
  all_filespecs.sort
end


def greeting
  puts <<~GREETING
      organize-av-files

      Enables the vetting of audio and video files. 

      For each file, plays it with mplayer, and prompts for what you would like to do 
      with that file, moving the file to one of the following subdirectories:

      * deletes
      * saves
      * undecideds

      This software uses mplayer to play audio files. Use cursor keys to move forwards/backwards in time.
      Press 'q' or 'ESC' to abort playback and specify disposition of that file.

      Run `man mplayer` for more on mplayer.

      Assumes all files specified are playable by mplayer.
      Creates subdirectories in the current directory: deletes, saves, undecideds.
      Logs to file '#{LOG_FILESPEC}'

  GREETING
end


def play_file(filespec)
  # If you have mplayer problems, remove the redirection ("2> /dev/null")
  # to see any errors.
  `mplayer #{filespec} 2> /dev/null`
end


def disposition_prompt(filespec)
  "\n\n#{filespec}:\ns = save, d = delete, u = undecided, q = quit: "
end


def get_disposition_from_user
  loop do
    response = $stdin.gets.chomp.downcase

    if response == 'q'
      exit
    elsif %w(s d u).include?(response)
      return {
          's' => 'saves',
          'd' => 'deletes',
          'u' => 'undecideds'
      }[response]
    else
      print "s = save, d = delete, u = undecided, q = quit: "
    end
  end
end


def log(filespec, destination_subdir)
  dest_abbrev = destination_subdir[0].upcase # 'S' for saves, etc.
  log_message = "#{dest_abbrev}  #{Time.now}  #{filespec}"
  `echo #{log_message} >> #{LOG_FILESPEC}`
end


def main
  check_presence_of_mplayer
  create_dirs
  puts greeting
  files_to_process.each do |filespec|
    play_file(filespec)
    print disposition_prompt(filespec)
    destination_subdir = get_disposition_from_user
    `mv #{filespec} #{destination_subdir}`
    log(filespec, destination_subdir)
  end
end


main

Technical Explorations

Enforcing Subclass Method Implementation in WiFi Wand Using TracePoint

Ruby’s inherited Hook: The Timing Problem and a TracePoint Solution in WiFi Wand

The Problem: Early Execution of inherited in WiFi Wand

Understanding the Timing Issue

Alternative Approaches (And Why They Fall Short)

Method 1: method_added Hook

Method 2: Manual Registration

The Solution: TracePoint to the Rescue

How the TracePoint Solution Works in WiFi Wand

WiFi Wand’s Method Design Philosophy

Internal Implementation Methods

Public API Methods

TracePoint Events the actual implementation methods that each OS must provide. They’re underscore-prefixed to indicate they’re internal implementations.

2. Public Methods (wifi_on, wifi_off, detect_wifi_interface, etc.)

TracePoint Events

Considerations and Limitations

Performance

Complexity

Maintenance

Real-World Benefits in WiFi Wand

Conclusion

Signaling Cursor AI Completion with a File Watcher and Screen Flasher

Quick Access: View and Download Scripts

Synchronous, Thread, and Fiber HTTP Requests in Ruby

Introduction

The Synchronous Approach

The Thread Approach

The Fiber Approach

Comparing the Performance Results

Fibers vs. Threads

Operating System Open File Handle Limits

JRuby

Large Request Counts

Conclusion

Addendum

AutoPilot, CoPilot, or AI Assistant?

AI Downsides

Resisting Oversimplification

The Optimum Balance

Organizational Responsibility

Conclusion

Installing International Language Keyboard Input Methods on Linux Mint 20 Cinnamon & Mate

Language Support

If…

Install IBus

Add IBus to the Startup Applications List

Configure IBus

Add a keyboard shortcut

Add input methods

Reboot and Test

The Language Panel Applet

Cautions

You’re done!

Add Bootstrap to Rails 6 with Two Shell Commands

Introduction

Even Simpler – A Patch

Adding the JavaScript Libraries

Testing that Bootstrap Works

Git’s Patch Support

Conclusion

The Case for Stabby Lambda Notation

The Stabby Lambda (->)

Stabby Notation as an Indicator of Preferred and Default Proc Type

->’s Picture-Like Notation

The Need for Visual Differentiation

Rubocop

Conclusion

lambdas Are Better Than procs

Arity (Argument Count) Checking Behavior Differences

Return Behavior Differences

A lambda is More Method-Like Than a proc

Conclusion

Footnotes

Ruby Enumerables Make Your Code Short and Sweet

Automate the Little Things Too

Integrating MPlayer and Ruby

The High Level View

Using Subdirectories

An Example

Ruby’s `inherited` Hook: The Timing Problem and a TracePoint Solution in WiFi Wand

The Problem: Early Execution of `inherited` in WiFi Wand

Method 1: `method_added` Hook

2. Public Methods (`wifi_on`, `wifi_off`, `detect_wifi_interface`, etc.)

The Stabby Lambda (`->`)

`->`’s Picture-Like Notation

A `lambda` is More Method-Like Than a `proc`