Bash and Paths on macOS

On macOS, there’s a little more than just the order of bash scripts messing with your path. Here’s an updated diagram showing the inside scoop.

I was helping a friend diagnose why a strange script-like string was appearing in their PATH whenever they logged in or created a new bash shell window. (Note: Catalina now defaults to using zsh, not bash.)

Bash has a number of different configuration files it checks, and some of those the user may have chained together. So far, even the best resource on the matter I’ve found doesn’t tell the whole story.

On Apple machines, bash walks through these set of scripts and invokes a path modification tool, too.

macOS Bash Scripts and Path Modifications (validated with macOS Mojave v10.14.6 on 2020-04-08)

In a nutshell, /etc/profile calls a utility called path_helper, and it reads a list from two places the /etc/paths file and everything in the /etc/paths.d directory.

My friend had thought the /etc/paths file was a script and not a list, and years ago had erroneously put an export PATH= statement in there. Because it came last (for his setup), he ignored the problem.

Our false start to correct is was upon seeing an export statement in the PATH itself, we assumed it was a faulty quoting problem in a script, so primarily went looking in the script files on the left side of the graph …and wasted what felt like hours doing so.

Only after using Bash’s return statement to short-circuit script executions and using a blunt hammer of renaming scripts out of execution’s way, and the problem still persisted, did deeper exploration go down the system resources.

I hope this diagram helps get others out of trouble.


UPDATE 2020-07-08: Bob Rudis caught a typo (the text said /etc/profile, where it should have said /etc/paths); he also shared a link to GNU’s Bash Startup Files.

CoffeeScript Bundle for TextMate (FIX for Command Not Found)

Installed HomeBrew to get Node.js, used the node package manager to get CoffeeScript, added a TextMate Bundle for CoffeeScript and got a “coffee: command not found” when doing a Command-B. WTF? Here’s how I fixed it.

I installed CoffeeScript on OS X using HomeBrew.

Got this error while using the CoffeeScript bundle for TextMate with the Command-B to Compile and Display JS.

/tmp/temp_textmate.CZvit9: line 12: coffee: command not found

In a new document, typing the word coffee and pressing Control-R to run it, didn’t work.

However, running coffee from the command line worked.
$ coffee -v
CoffeeScript version 1.1.2

Solution: Add /usr/local/bin to the TextMate path.

JSON and Undefined Object Properties

Ran into an interesting case where the JavaScript object coming out of JSON wasn’t the same as the JavaScript object encoded going in.

When are JSON.parse() and JSON.stringify() not inverses of each another?

Here’s when.

In JavaScript it’s possible to define an object that has an undefined property:

var o = {
    a: "Hello",
    b: undefined
};

It’s then possible to iterate over all the objects properties and see the values as well.

for (var p in o) {
    alert( p + ": " + o[p] );
}

This will show “a: Hello” and “b: undefined”.

Now, to JSON-ify the object and reconstitute it back.
o = JSON.parse( JSON.stringify(o) );

Doing the for-loop again will reveal the object now only has the “a” property (the value is still “Hello”).

Try it.

While I get it, I’m not sure I like it. Ideally, I’d like anything coming out of JSON to be the same as anything going into it.

Assume this object:

var me = {
    mySiblingsGender: undefined
};

This is one of those cases where several subtleties rear their head on edge cases.

First, a known value for mySiblingsGender might be male or female. A null would indicate there is no gender, perhaps by a horrible vegetable peeler accident. (Oh my!) But an undefined value indicates there is one, we just don’t know what it is.

Removing the property completely instead conveys that there is no sibling (absence) — something totally different than null (not having any) or undefined (unknown).

While it’s splitting hairs, if not kindling for religious coding wars, it’s useful to know that JSON-out may not always be the same as JSON-in.

Un-denying a Denied Host

A great guy named Nicholas put out an easy to use script to unban an ip address that’s been banned by denyhosts. Here’s me gushing with praise since I could be lazy and not have to write it, along with the script so you can benefit from his contribution as well.

A big thanks to Nicholas Avenell, who solved a problem I’ve been having for a while: How to remove a host that denyhosts has banned. Yes, the FAQ for denyhosts has the manual steps listed, but Nicholas scripted it. And it works great.

Copy Nicholas’s script into a file, give it execute permissions, and run it with root privileges passing in the IP address you’d like to unban:

#/bin/sh
REMOVE=$1
/etc/init.d/denyhosts stop
cd /var/lib/denyhosts
for THISFILE in hosts hosts-restricted hosts-root hosts-valid users-hosts;
do
mv $THISFILE /tmp/;
cat /tmp/$THISFILE | grep -v $REMOVE > $THISFILE;
rm /tmp/$THISFILE;
done;
mv /etc/hosts.deny /tmp/
cat /tmp/hosts.deny | grep -v $REMOVE > /etc/hosts.deny;
rm /tmp/hosts.deny
/etc/init.d/denyhosts start

Nicholas, you rule. Thanks.

C# Must Haves

Resources for quickly getting up to speed on C#.

This is the page of C# resources that I wish I had when I got started.

Visual Studio 2010Snag a copy of Visual Studio, whether it’s the free Express version, or one of the uber-featured ones.

Reactive Extensions
In addition to .NET, snag the Reactive Extensions (Rx) library.


Expression Blend 4If you’re able to, grab a copy of Expression Blend as well for graphical layouts. Not mandatory, but it helps.


And you’ll want these books:

C# 4.0 in a Nutshell: The Definitive ReferenceWPF 4 UnleasedLinq in ActionEntity Frameworks

Mysterious Copyright

This is clearly one of those things I did to myself as a good idea, then forgot about, only to be plagued by it later.

I noticed that all of my photographs on my camera were reporting a copyright with a 2009 year inside the exif data.

I’ve been unable to figure out where it was coming from, resorting to exiftool to remote it.

My natural thought was that perhaps it was some preference in a photo editing tool or a geospatial locator tool. But, no. Turns out I did it to myself.

The Canon EOS Utility has a nifty ability to include a value for the Copyright tag. And about a year ago when I tethered it to the computer, I must have noticed this and set it to some precanned value that includes the year.

It looked something like this:
Copyright (c) 2009 by Walt Stoneburner, All Rights Reserved.

And ever since then, my photos were stamped with that value. Which was fine, back in 2009.

Fixing the problem was as simple as tethering the camera again and firing up Canon EOS Utility. It also gave me an opportunity to update the firmware.

Strange copyright exif data: mystery solved.

Find and Replace in Word using C# .NET

Solution to how to do a global search and replace in MS-Word, including across floating text objects, in C#/.NET.

Heads up, this article contains high quantity of geek content. Non-geeks should move along.

I’ve been trying to use Microsoft.Office.Interop.Word to perform a global bulk search and replace operations across an entire document. The problem was, however, if a document contained a floating text box, which manifested itself as a shape object of type textbox, the find and replace wouldn’t substitute the text for that region. Even using Word’s capability to record a macro and show the VBA code wasn’t helpful, as the source code in BASIC wasn’t performing the same operation as inside the Word environment.

What I wanted was a simple routine to replace text anywhere inside of a document. If you Google for this you’ll get the wrong kind of textbox, the wrong language, people telling you not to use floating textboxes, and all kinds of weird story iterators.

One site seemed to have the solution; many kind thanks to Doug Robbins, Greg Maxey, Peter Hewett, and Jonathan West for coming up with this solution and explaining it so well.

However, the solution was in Visual Basic for Applications, and I needed a C# solution for a .NET project. Here’s my port, which works with Office 2010 and Visual Studio 2010 C#/.NET 4.0. I’ve left a lot of redundant qualifiers and casting on to help people searching for this article.

using System;
using System.Collections.Generic;
using System.ComponentModel;
using Microsoft.Office.Interop.Word;

// BEGIN: Somewhere in your code
Application app = null;
Document doc = null;
try
{
  app = new Microsoft.Office.Interop.Word.Application();

  doc = app.Documents.Open(filename, Missing, Missing, Missing, Missing, Missing, Missing, Missing, Missing, Missing);

  FindReplaceAnywhere(app, find_text, replace_text);

  doc.SaveAs(outfilename, Missing, Missing, Missing, Missing, Missing, Missing, Missing, Missing, Missing);
}
finally
{
  try
  {
      if (doc != null) ((Microsoft.Office.Interop.Word._Document) doc).Close(true, Missing, Missing);
  }
  finally { }
  if (app != null) ((Microsoft.Office.Interop.Word._Application) app).Quit(true, Missing, Missing);
}
// END: Somewhere in your code             



// Helper
private static void searchAndReplaceInStory(Microsoft.Office.Interop.Word.Range rngStory, string strSearch, string strReplace)
{
    rngStory.Find.ClearFormatting();
    rngStory.Find.Replacement.ClearFormatting();
    rngStory.Find.Text = strSearch;
    rngStory.Find.Replacement.Text = strReplace;
    rngStory.Find.Wrap = WdFindWrap.wdFindContinue;

    object arg1 = Missing; // Find Pattern
    object arg2 = Missing; //MatchCase
    object arg3 = Missing; //MatchWholeWord
    object arg4 = Missing; //MatchWildcards
    object arg5 = Missing; //MatchSoundsLike
    object arg6 = Missing; //MatchAllWordForms
    object arg7 = Missing; //Forward
    object arg8 = Missing; //Wrap
    object arg9 = Missing; //Format
    object arg10 = Missing; //ReplaceWith
    object arg11 = WdReplace.wdReplaceAll; //Replace
    object arg12 = Missing; //MatchKashida
    object arg13 = Missing; //MatchDiacritics
    object arg14 = Missing; //MatchAlefHamza
    object arg15 = Missing; //MatchControl

    rngStory.Find.Execute(ref arg1, ref arg2, ref arg3, ref arg4, ref arg5, ref arg6, ref arg7, ref arg8, ref arg9, ref arg10, ref arg11, ref arg12, ref arg13, ref arg14, ref arg15);
}

// Main routine to find text and replace it,
//   var app = new Microsoft.Office.Interop.Word.Application();
public static void FindReplaceAnywhere(Microsoft.Office.Interop.Word.Application app, string findText, string replaceText)
{
    // http://forums.asp.net/p/1501791/3739871.aspx
    var doc = app.ActiveDocument;

    // Fix the skipped blank Header/Footer problem
    //    http://msdn.microsoft.com/en-us/library/aa211923(office.11).aspx
    Microsoft.Office.Interop.Word.WdStoryType lngJunk = doc.Sections[1].Headers[WdHeaderFooterIndex.wdHeaderFooterPrimary].Range.StoryType;

    // Iterate through all story types in the current document
    foreach (Microsoft.Office.Interop.Word.Range rngStory in doc.StoryRanges)
    {

        // Iterate through all linked stories
        var internalRangeStory = rngStory;

        do
        {
            searchAndReplaceInStory(internalRangeStory, findText, replaceText);

            try
            {
                //   6 , 7 , 8 , 9 , 10 , 11 -- http://msdn.microsoft.com/en-us/library/aa211923(office.11).aspx
                switch (internalRangeStory.StoryType)
                {
                    case Microsoft.Office.Interop.Word.WdStoryType.wdEvenPagesHeaderStory: // 6
                    case Microsoft.Office.Interop.Word.WdStoryType.wdPrimaryHeaderStory:   // 7
                    case Microsoft.Office.Interop.Word.WdStoryType.wdEvenPagesFooterStory: // 8
                    case Microsoft.Office.Interop.Word.WdStoryType.wdPrimaryFooterStory:   // 9
                    case Microsoft.Office.Interop.Word.WdStoryType.wdFirstPageHeaderStory: // 10
                    case Microsoft.Office.Interop.Word.WdStoryType.wdFirstPageFooterStory: // 11

                        if (internalRangeStory.ShapeRange.Count > 0)
                        {
                            foreach (Microsoft.Office.Interop.Word.Shape oShp in internalRangeStory.ShapeRange)
                            {
                                if (oShp.TextFrame.HasText != 0)
                                {
                                    searchAndReplaceInStory(oShp.TextFrame.TextRange, findText, replaceText);
                                }
                            }
                        }
                        break;

                    default:
                        break;
                }
            }
            catch
            {
                // On Error Resume Next
            }

            // ON ERROR GOTO 0 -- http://www.harding.edu/fmccown/vbnet_csharp_comparison.html

            // Get next linked story (if any)
            internalRangeStory = internalRangeStory.NextStoryRange;
        } while (internalRangeStory != null); // http://www.harding.edu/fmccown/vbnet_csharp_comparison.html
    }

}

Let me know if it worked for you; bug fixes and enhancements welcome.

Seeing in Black and White

I just had an interesting thing happen: I saw in black and white. That’s what my brian actually saw with the unaided eye. Here’s the cool part, I tell you how I reproduced it. It was like nothing I’d experienced before. It was beautiful.

A few moments ago, I just had a very interesting and unique experience. I saw in black’n’white. I’d never had this happen before in my life. First I’ll describe the experience, then how I did it, which, curiously enough was repeatable.

“It was clearly not imagination… It was a greyscale world that I physically saw, like a black and white movie, but a zillion times sharper, far more dynamic range, and in 3D.”
With the totally unaided eye, my brain saw my surroundings, in broad daylight, in black and white. The only exception was that objects which were normally bright red had an ever slight red hue to them, but it was only brilliant red objects that did this.

The effect lasted about 7-10 seconds in duration before the color faded back in, almost as if the saturation was being brought up from near nothing to normal.

To convey the effect, this is much like the image I saw:

Seeing in Black and White

When I moved my eyes to look at other parts of the scene, the effect diminished, but if I kept focusing on one spot, like a child’s staring contest, the effect would hold longer. This was very much the inverse of the behavior of an after-image, where if you stay still it fades, but if you rapidly blink, it returns.

The black and white effect composed of the entire field of view. And as it gently faded back to normal, it affected more of the center of the field of view first:

Seeing in Black and White

It only took 2-3 seconds for normal color to return. There was no pain or any form of discomfort before, during, or after. I’m in very good health.

It was as if the signals from the cones were being ignored by the brain, but the signals from the rods were fine. I remember that the detail was astounding, and that the tone of the grass was very similar to the sky, though I’ve been unable to represent that as closely as I’d like in my photographic simulation.

I’m certain you’ve personally woken up in the morning and upon your eyes adjusting to the light have seen the image fade in, not focus, but as your brian assembles bits of the information into meaningful images, like it’s adapting to light after not being exposed to it for a long while.

Here’s how I did it.


I’m going to err on the side of giving too much information, some that might not be relevant, primarily because I don’t know what actually caused it. However, I was able to recreated it, on demand, several minutes later by experimentation; the effect lasted even longer. It was wondrous.

Being inside for the better part of the morning, I figured I’d go outside an lay in the sun for a few minutes. So, I lay down on my back on our driveway which has a slight incline. It was about thirty minutes past noon, and the sun was slightly overhead just off center to my right, enough that it was still bright enough that just closing my eyes was uncomfortable, so I criss-crossed my arms over them to put them in shadow, though I could still tell it was very bright out with my eyes closed. The weather was 89°F, I was in direct light, and there were few clouds in the sky.

I rested this way for about 10 minutes, and I did so just to the point where my eyes were fully relaxed and no longer concerned about the brightness of the light though my eyelids. Also, I wasn’t quite drifting off, but relaxed as you might be just taking in warmth of a nice day at the beach.

What led to the discovery was that I heard a car drive by and so I sat up quickly, opening my eyes. Two things stood out. One, this gave me a slight head-rush, though I’d describe it weak at best. Two, my eyes had not adjusted to the light fully. And, although while bright, it wasn’t uncomfortable, there was no blinding whitewash, no pain, and no caused for squinting required — I was looking away from direct light.

That’s when I noticed the scene seemed extremely washed out and monochromatic; it looked like a black and white photograph.

Thinking my mind or eyes were playing tricks on me, I moved my eyes, but the effect lasted longer than a second, though faded as I looked at more “new” material in my field of attention.

The reds came rushing back in first, with greens right after. It wasn’t one color than another, it was overlapped. I became visually more aware of reds, as that happened, greens started replacing the grey tones as well, and by the time greens were normal, the other colors like purples and blues from the nearby flowers in our garden were already present.

This part will be hard to describe because there’s no English equivalent for it, but it wasn’t like I was seeing in black and white, but rather the absence of color.

I know that sounds identical from a logic standpoint, but the perception was an absence of something, not the presence of something. Intellectually, I knew there had to be color, I just wasn’t seeing it.

It was a greyscale world that I physically saw, like a black and white movie, but a zillion times sharper, far more dynamic range, and in 3D.

It was clearly not imagination, nor dream, it was very real and perceptible.

That’s what made me want to try and repeat it.


Though this time around, if I could get it to work, I’d plan a more scientific excursion. So I selected an area of our yard with brilliant colors, our flower garden, which would be my baseline.

First, I waited about 5 minutes and looked at the area, taking in all the things I should be expecting to see. The area was already in normal color after the effect had faded long ago, but I wanted to be sure.

Seeing in Black and White

So, I laid back down with the intention of trying to get the head rush. I shielded my eyes in shadow, and waited for them to relax and get comfortable, and then waiting a moment, quickly sat up and stared at the area I knew had colorful flowers, a bright red car, and tons of green grass. I made sure I would focus only in one area, trying to stare out to infinity, much as you’d do for those 3D posters, though I was more trying to keep my eyes in a relaxed state because I wanted them in focus.

The preparation this time around took merely a minute or two. The head-rush was again weak from sitting up quickly. I turned in the direction I had planned and opened my eyes.

It’d worked.

The effect returned just as before, but lasted this time about 20 seconds and the effect was just as strong until I couldn’t help myself and look at the astounding detail in scene around me, which caused the effect to fade.

Since I had more time to study the scene, this is when I actually noticed the reds within the black’n’white image in my head. The B/W effect seemed more pronounced when I didn’t move my eyes, relaxing them.

When I turned to look at other things, color would seep in, and if I held my eyes relaxed in the same spot, the color would fade back out partly, much the same way as you can make objects in your blind spot vanish by holding your vision still long enough. That same kind of fading away was what I saw, but with color.

I suspect the effect is caused less by the head rush and more by the eyes being exposed and conditioned to bright light (even with the eyes closed, or maybe the red light through the eyelids saturates the cones like a red filter, though I did not see a green after image). When the eyes are opened, there’s a whole rush of visual information, and I suspect the brain is compensating for the overload by taking in shapes, detail, and tone and then overlaying color after the signal settles. I wasn’t aware that visual processing of color was an independent process.

This led me to two very interesting side thoughts.

One, I’ve always wondered if while under hypnosis people really saw things but said they didn’t, or whether their honestly perceived it. I now know it’s possible to perceive things differently than the visual input as actually providing.

Two, the black and white image was astonishingly detailed in greyscale, much like an Ansel Adams image. Being able to produce this effect on demand to view how a scene might be photographed in black and white is a fantastic tool to have in one’s photographic arsenal.

I hope science doesn’t declare this is bad for you, because I’m going to do it again!

[UPDATE: I can’t get it to happen inside, seems bright sunlight is required.]

FIX: undefined symbol: apr_ldap_ssl_init

Did an update to Ubuntu Jaunty and Apache stopped working with the message “undefined symbol: apr_ldap_ssl_init”. This post is how I fixed it.

This is a geek entry for resolving the problem:

* Restarting web server apache2
/usr/sbin/apache2: symbol lookup error: /usr/sbin/apache2: undefined symbol: apr_ldap_ssl_init [fail]

Non-geeks will want to move along…
Continue reading “FIX: undefined symbol: apr_ldap_ssl_init”