Unreal Expectations

That actually worked??!??

XMPPFramework / iOS / libidn / -all_load failure

As I integrated the XMPPFramework it requires the libidn library for a couple of methods it uses. The latest version of libidn that the XMPPFramework included as of this writing is 1.15.

Unfortunately, this version seems incompatible with the “Other Linker Flag” -all_load that other libraries seem to want to use. The all_load linker flag appears be a work around for a linker bug that pops up when object files within a static library only contain a category and no classes.

From the source: http://developer.apple.com/mac/library/qa/qa2006/qa1490.html

IMPORTANT: For 64-bit and iPhone OS applications, there is a linker bug that prevents -ObjC from loading objects files from static libraries that contain only categories and no classes. The workaround is to use the -all_load or -force_load flags. -all_load forces the linker to load all object files from every archive it sees, even those without Objective-C code. -force_load is available in Xcode 3.2 and later. It allows finer grain control of archive loading. Each -force_load option must be followed by a path to an archive, and every object file in that archive will be loaded.

Interestingly, most coding conventions will create header and implementation files for categories that are just the categories; hence exposing the defect. Some library developers have come up with a simple workaround so that the force_load or all_load flags are not required. The workaround is a simple macro that is added to the category implementation file that just creates a class so that the bug is not exposed. Don’t know if it originated in Three20, but this is first place I came across it. RestKit also implemented the macro.

Unfortunately, other library developers have not yet implemented this, and still require the all_load flag to be used.

The use of the all_load flag has been causing the XMPPFramework’s use of libidn to fail to link. In part this appears to be the use of older libidn that is included as fat binary, static library. Use of the all_load flag often causes this linker error:

Undefined symbols for architecture i386:
“_libintl_bindtextdomain”, referenced from:
_idna_strerror in libidn.a(strerror-idna.o)
_pr29_strerror in libidn.a(strerror-pr29.o)
_punycode_strerror in libidn.a(strerror-punycode.o)
_stringprep_strerror in libidn.a(strerror-stringprep.o)
_tld_strerror in libidn.a(strerror-tld.o)
“_libintl_dgettext”, referenced from:
_idna_strerror in libidn.a(strerror-idna.o)
_pr29_strerror in libidn.a(strerror-pr29.o)
_punycode_strerror in libidn.a(strerror-punycode.o)
_stringprep_strerror in libidn.a(strerror-stringprep.o)
_tld_strerror in libidn.a(strerror-tld.o)
“_libiconv”, referenced from:
_str_cd_iconv in libidn.a(striconv.o)
_mem_cd_iconv in libidn.a(striconv.o)
“_libiconv_open”, referenced from:
_str_iconv in libidn.a(striconv.o)
“_libiconv_close”, referenced from:
_str_iconv in libidn.a(striconv.o)
ld: symbol(s) not found for architecture i386
clang: error: linker command failed with exit code 1 (use -v to see invocation)

The iconv errors can be fixed because the iOS framework includes libiconv.a. It does not include libintl.a. After downloading version 1.25 of libidn and getting it to install on iOS 5.0, I was able to get rid of errors related to intl.

Here are the commands I used to compile libidn.1.25 and create a fat binary with arm6, arm7, and i386 architecture files.


$ ./configure --host=arm-apple-darwin --disable-shared CC=/Developer/Platforms/iPhoneOS.platform/Developer/usr/llvm-gcc-4.2/bin/llvm-gcc-4.2 CFLAGS="-arch armv7 -fmessage-length=0 -pipe -std=c99 -Wno-trigraphs -fpascal-strings -O0 -Wreturn-type -Wunused-variable -Wunused-value -isysroot /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS5.0.sdk -fvisibility=hidden -gdwarf-2 -mthumb -miphoneos-version-min=4.0 " CPP=/Developer/Platforms/iPhoneOS.platform/Developer/usr/llvm-gcc-4.2/bin/llvm-cpp-4.2 AR=/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/ar
$ make clean
$ make
$ cp lib/.libs/libidn.a libidn-arm7.a

$ ./configure --host=arm-apple-darwin --disable-shared CC=/Developer/Platforms/iPhoneOS.platform/Developer/usr/llvm-gcc-4.2/bin/llvm-gcc-4.2 CFLAGS="-arch armv6 -fmessage-length=0 -pipe -std=c99 -Wno-trigraphs -fpascal-strings -O0 -Wreturn-type -Wunused-variable -Wunused-value -isysroot /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS5.0.sdk -fvisibility=hidden -gdwarf-2 -mthumb -miphoneos-version-min=4.0 " CPP=/Developer/Platforms/iPhoneOS.platform/Developer/usr/llvm-gcc-4.2/bin/llvm-cpp-4.2 AR=/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/ar
$ make clean
$ make
$ cp lib/.libs/libidn.a libidn-arm6.a

$ ./configure --host=i686-apple-darwin --disable-shared CC=/Developer/Platforms/iPhoneSimulator.platform/Developer/usr/llvm-gcc-4.2/bin/llvm-gcc-4.2 CFLAGS="-arch i386 -fmessage-length=0 -pipe -std=c99 -Wno-trigraphs -fpascal-strings -O0 -Wreturn-type -Wunused-variable -Wunused-value -isysroot /Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator5.0.sdk -fvisibility=hidden -gdwarf-2 -mthumb -miphoneos-version-min=4.0 " CPP=/Developer/Platforms/iPhoneSimulator.platform/Developer/usr/llvm-gcc-4.2/bin/llvm-cpp-4.2 AR=/Developer/Platforms/iPhoneSimulator.platform/Developer/usr/bin/ar
$ make clean
$ make
$ cp lib/.libs/libidn.a libidn-i386.a

$ lipo -create libidn-i386.a libidn-arm6.a libidn-arm7.a -output libidn.a

After importing the new libidn,a, I still received the linking errors with libiconv. Added that to the library link list and afterwards all worked.

Multiple Language Meta Descriptions / One Page

During a post implementation design review today, the design and implementation failed to meet the basic one requirement of the enhancement. The requirement was to ensure the site in question would respond to Katakana characters when searched through google.co.jp. The english title of the site is returned quite high, but when using the Katakana characters it fails.

The site in question is english only and so it mostly makes sense that it fails in google search results for Katakana. So the requirement was how to make the site appear utilizing Katakana.

Technical Solution

The developer went off and designed and implemented a solution. The solution relied upon the Accept-Language header in HTTP request to load a resource file and return a Katana localized version of the title and description if the preferred language is Japanese. Logically, seems to make sense. Change the default language on the browser and get the localized content.

Except there is a hole in the solution you can drive a bus through. When the googlebot crawler indexes sights it doesn’t have different crawlers for google.com versus google.co.jp versus other localized google sites. It has a single googlebot that indexes pages and doesn’t provide a language header. With the change, google would not appear to have interpreted the page in any way different, which is the case because it doesn’t get any different HTML.

From the Google Webmaster Blog we know content negotiation is bad.

@Simon Content negotiation is bad because search engines will only be able to see one of the versions (and might never see the other one). Also, many users (including me) don’t like it when they click on a search result that is not shown in the same language as their query (and the snippet) (actually, I personally don’t just not like it, it makes me quite mad :-)).

SEO Solution

Examining that the technical delivery failed, I backed up from there on the original solution concept.

The solution concept was to just add the Katana characters after the title and as a second description. According the the following information who experimented upon other results found, as expected, that google ignores multiple meta-description tags. It points to other article about partial success in google summaries using more relevant sentence from multi line description.

So the point of this is to try and see how multi languages will work in context of a description. Will update with results.

Update 1

The first launch failed miserably because the theme is not setting or pulling in the meta title or description fields.

Install Your Own URL Shortener

A while back I acquired domain, zod.im with intent of setting up my own shortened URL service. While there is nothing wrong with the current offerings by bit.ly or TinyURL, I was academically intrigued by the notion of setting this up.

YOURLS

I went with open source package YOURLS. It was pretty easy to setup and has a nice interface. The documentation didn’t provide me everything I needed, but what it didn’t provide was easily discovered. Examples:

I don’t install MySQL databases on a daily, weekly, or monthly basis, so I had to go look that back up since the commands weren’t included directly in the setup documentation.
I front nginx as the web server and chose to look up changes to nginx configuration as the yourls installer provides just content for .htaccess
I use WordTwit for notifications of posts on this blog to send to Twitter. The current version did not have yourls support, but looks like the next Pro version might have more pluggable solution. So made updates to plugin to use this now.

Installation

Pre-Requisites

This assumes the starting point of installation already has nginx installed with fastcgi able to interpret php.
This also assumes MySQL is installed and operational.

MySQL

Create the database


mysql> CREATE DATABASE yourlsdb;

Create the user & Grant


mysql> CREATE USER 'yourlsdbuser'@'localhost' IDENTIFIED BY 'password';

mysql> GRANT ALL PRIVILEGES ON yourlsdb.* TO yourlsdbuser@localhost;

Install YOURLS

Download the code

Check here for the latest version.


$ wget http://code.google.com/p/yourls/downloads/detail?name=yourls-1.5.zip

Unzip the code in directory you want as root


$ sudo mkdir -p /var/www/zod.im/web
$ cd /var/www/zod.im/web/
$ unzip <path to>/yourls-1.5.zip

Update the configuration

The following is the minimum settings that need to be changed for the private linking to work. Read through the configuration for other changes to consider.

Copy the sample configuration


$ cp includes/config-sample.php user/config.php

Update the database settings

Use vi or whatever your favorite editor is to edit the new file and follow the information in the file.
Change the database settings.


...
define( 'YOURLS_DB_USER', 'dbuser' );

/** MySQL database password */
define( 'YOURLS_DB_PASS', 'dbpassword' );

/** The name of the database for YOURLS */
define( 'YOURLS_DB_NAME', 'yourls' );

/** MySQL hostname */
define( 'YOURLS_DB_HOST', 'localhost' );
...

Update the base site URL


/** YOURLS installation URL, no trailing slash */
define( 'YOURLS_SITE', 'http://example.com' );

Update the cookie hash

As the comment shows go here to get a new random hash.


/** A random secret hash used to encrypt cookies. You don't have to remember it, make it long and complicated. Hint: copy from http://yourls.org/cookie **/
define( 'YOURLS_COOKIEKEY', 'qQ4KhL_pu|s@Zm7n#%:b^{A[vhm' );

Update the username and password

The username and passwords are in the configuration file with password in clear text. Since this isn’t secure, I recommend not using a password that works anywhere else.


/**  Username(s) and password(s) allowed to access the site */
$yourls_user_passwords = array(
        'username' => 'password',
        'username2' => 'password2'      // You can have one or more 'login'=>'password' lines
        );

Setup Nginx for Rewrite

Edit the nginx configuration for the domain. The configuration of YOURLS provides an .htaccess file with rewrite rules which is fine for Apache, but not so well with nginx.



server {
  server_name www.zod.im www.zodim.com;
  rewrite ^(.*) http://zod.im$1 permanent;
}

server {
    listen   80;
    server_name zod.im;

    access_log /var/log/nginx/zod.im.access.log;
    error_log /var/log/nginx/zod.im.error.log;

    root   /var/www/zod.im/web/;
    index  index.php index.html;

    error_page    404    /404.php;

  location / {
    # If the file exists as a static file serve it directly without
    # running all the other rewite tests on it
    if (-f $request_filename) {
      expires max; 
      break; 
    }

    if (!-f $request_filename){
       set $rule_0 1$rule_0;
    }
    if (!-d $request_filename){
       set $rule_0 2$rule_0;
    }
    if ($rule_0 = "21"){
       rewrite ^/.* /yourls-loader.php last;
    }
  }
  location ~ .php$ {
    set  $script     $uri;
    set  $path_info  "";

    if ($uri ~ "^(.+\.php)(/.+)") {
      set  $script     $1;
      set  $path_info  $2;
    }

    include /etc/nginx/fastcgi_params;
    fastcgi_pass 127.0.0.1:5555;
    fastcgi_index index.php;

    fastcgi_param SCRIPT_FILENAME /var/www/zod.im/web$fastcgi_script_name;
    fastcgi_param  PATH_INFO        $path_info;
    fastcgi_intercept_errors on;

    root /var/www/zod.im/web;
  }

Restart nginx

Go through setup

Open browser and go to the admin page, login and go through initial setup process.

http://example.com/admin/

You get two screens to see. First is initial welcome with button to install. Click this and get the second telling you it has created the database tables.

The second screen may complain about not being able to write the .htaccess file, but since nginx is used, that can easily be ignored.

Conclusion

The process to create your own personal shortening service can take a little as little as 15 minutes with YOURLS. It will probably take longer to figure out your short domain name and acquire it.

Eye-Fi Upload via Mac OS X Internet Sharing

I occasionally find the desire to upload images via Eye-Fi while in a location that doesn’t have Wi-Fi, but my computer does have wired Internet access. Let’s say for lack of a better example location this could be a late 90’s remodeled hotel. In those instances, I feel using the Internet Sharing of Mac OS X seems to be an easy solution. Yeah, not so much. Most of the times I have used Internet Sharing, it worked great. However, those cases tended to be using other Apple hardware such as iPhone or iPad when I needed to trick it into thinking the device has a Wi-Fi connection when one was not readily available. However, using non-Apple devices seem to be a bit more tricky.

Using Internet Sharing in Max OS X utilizing the Airport for incoming tracking sets the Wi-Fi Network up in Infrastructure mode. The Mac is able to act as Router and provide NAT-based private IP addresses to clients. It also supports WEP, both 40 bit and 128 bit. It does not support WPA or WPA2 when the airport acts as access point.

reply_threshold_seconds

There are ample examples of findings of a bug in Mac OS X that a defect in the default configuration of DHCP server when Internet Sharing is enabled that delays the response to DHCP requests beyond what some client devices find acceptable. This causes the device to think it can’t receive an address and report an error. It turns out if this setting is decreased more devices can connect.

To fix:

Go to System Preferences -> Sharing and enable Internet Sharing.
This is necessary because by default, the starting and stopping of Internet Sharing creates and deletes the /etc/bootpd.plist file.
Open terminal and create a copy of the bootpd.plist file.
```
> cp /etc/bootpd.plist /tmp/bootpd.plist
```

Open the /tmp/bootpd.plist file in editor (vi, TextEdit, TextWrangler, UltraEdit) and update the reply_threshold_seconds setting from 4 to 0.

Before:


        ...
        <key>reply_threshold_seconds</key>
        <integer>4</integer>
        ...

After:


        ...
        <key>reply_threshold_seconds</key>
        <integer>0</integer>
        ...

Stop the Internet Sharing.
This is necessary to replace the /etc/bootpd.plist file. Doing the next step before this one will cause the file to disappear the next time Internet Sharing is turned off.
From the terminal copy the edited file back to /etc directory.
```
> sudo cp /tmp/bootpd.plist /etc/
```
Start Internet Sharing.
Once the /etc/bootpd.plist file exists different from the default, Starting/Stopping the service does not seem to affect the file any longer.

Assuming you do not enable WEP, it is likely any client device can now connect to the new wireless access point.

To WEP or Not To WEP

While WEP is accepted as broken and can easily be hacked to determine the password, the use of WEP and requirement of password does keep the average WiFi joe from hopping on to your network at their whim. There was a survey published recently that reported that while the vast majority of WiFi owners locked down their routers to prevent others from using their network, the people had little compunction about using other peoples network when found to be open. Once or twice I have been in a neighborhood and needed network access on laptop. It was handy to find open network I could VPN through. So, I would recommend using WEP when nothing else is available.

Unfortunately, this proved fruitless with Eye-Fi. There are plenty of examples on how to fix this for Windows-based machines that had problems connecting. In trying to come up with working solution for Eye-Fi I first determined whether another device, such as Windows XP laptop could connect to again see whether it was non-Apple issue in whole or Eye-Fi specific issue. What the WEP based troubleshooting all stated for this that when using an alphanumeric password, it is likely that different clients are encoding the plain text differently. So, it is best to use a hex password which doesn’t require encoding process to use.

To use hex password, you can’t just enter the 10 or 26 hex characters into the password field in Mac OS X. Rather you need to prefix it with ‘$’ (dollar sign) character. The character explicitly tells the Internet Sharing control to use the values as hex.

Example: 40 Bit Password. Enter the following in the Mac OS X password box:


    $073E2ABF12

Enter the following in Windows or other client:

073E2ABF12

The 128 Bit version is much the same except using 26 hex character string.

Utilizing this approach I got the Windows laptop to connect, but the Eye-Fi card remain unable to fully connect. Except that I finally got a different response setting up the card in Eye Fi Center than previously. Specifically, most previous failures had the Connect button spinning into Idle Card state. This time, the error message stated to check whether the router was filtering MAC addresses and to allow the Eye-Fi card’s address if that was the case.

One last trick will be try setting the card up on Mac Mini connecting through to the Mac Book Pro in case there is any issues with circular connectivity. However, I don’t think this is the case since I added the network to the Eye-Fi card by stating the network wasn’t present. This should add the network to the card and if setup properly allow the card to connect in the camera. This still didn’t happen, so don’t have too much faith this is a circular/dual addressing issue.

No WEP, but locked with MAC Filtering

In the end, I want this to work even if I don’t have the illusion of security. So, go back to open access, but I don’t wan’t any Joe Blow connecting to my laptop. To prevent this then, means adding some MAC address filtering to allow allow known devices to connect to the Mac Book.

The problem is that the Internet Sharing preferences pane does not have an option for setting up MAC address filters. However, the bootpd configuration file does. Since I have already modified the /etc/bootpd.plist to change the reply_threshold_seconds it can be modified to enable MAC address filtering. To do this, just add the key allow with a string array value between the Subnets and bootp_enabled keys.


        ....
        <key>allow</key>
        <array>
                <string>0:17:43:a3:2d:45</string>
        </array></strong>
        ...

The MAC address entered is the one for the Eye-Fi card. In this case, since I got the error message with the address prominently displayed, it was pretty easy to find. Otherwise, right clicking on the card in Eye-Fi Center will list the MAC address in the menu with option to copy it.

Update the /etc/bootpd.plist file and restart the service. Now Eye-Fi could connect, but iPhone and Windows XP laptops could not. So all appears well enough.

Security

I never like not enabling some encryption even if it is just WEP. Even when on public hot spots, I like to use VPN to prevent local traffic sniffing.

I kind of figure in this case, the only thing being transferred is photos which generally end up in public locations anyway. If I need other devices somewhat secured, it is enough to make the changes, add the new device to MAC address filters and restart the filter.

What’s Next

I will keep trying to see if I can eventually get the card working with WEP through Mac OS X / Internet Sharing. One thing is to check if it is local or just as problematic same machine syndrome.

Full /etc/bootpd.plist File


<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Subnets</key>
  <array>
    <dict>
      <key>_creator</key>
      <string>com.apple.nat</string>
      <key>allocate</key>
      <true/>
      <key>dhcp_domain_name_server</key>
      <string>10.0.2.1</string>
      <key>dhcp_router</key>
      <string>10.0.2.1</string>
      <key>lease_max</key>
      <integer>86400</integer>
      <key>lease_min</key>
      <integer>86400</integer>
      <key>name</key>
      <string>10.0.2</string>
      <key>net_address</key>
      <string>10.0.2.0</string>
      <key>net_mask</key>
      <string>255.255.255.0</string>
      <key>net_range</key>
      <array>
        <string>10.0.2.2</string>
        <string>10.0.2.254</string>
      </array>
    </dict>
  </array>
  <key>allow</key>
  <array>
    <string>0:17:43:a3:2d:45</string>
  </array>
  <key>bootp_enabled</key>
  <false/>
  <key>detect_other_dhcp_server</key>
  <integer>1</integer>
  <key>dhcp_enabled</key>
  <array>
    <string>en1</string>
  </array>
  <key>reply_threshold_seconds</key>
  <integer>0</integer>
</dict>
</plist>

Impressions of Sony Dash

For Christmas I received the Sony Dash as present from my wife. Having worked with computers, networking, and all sorts of technology in general I am both amazed and dismayed by Sony Dash at the same time.

Since this article can live for a long time and the Sony product team does seem to making monthly/bi-monthly updates, some of these comments may become outdated.

Amazed

Portal Integration
The device uses a web application to help install and configure the apps that run on the device. The portal makes it relatively easy to find apps that are available, but if the number grows, it will need to address some things to make it easier to find useful ones.

Unfortunately, a lot of the configuration is via Flash, which makes sense given the apps on the Dash itself are built in Flash. This makes it impossible to use with iPad. If the Dash lives in the bedroom, it is easier to configure it with the iPad rather than laptop that usually stays downstairs. It’ll probably work with Android tablets, but don’t currently have one of those.

Netflix
We don’t have a TV in our bedroom, and while the prospect of watching a movie on 7 inch screen isn’t going to be the bedroom home theater of choice, it is still an interesting device to have Netflix access on. It is able to access my Wireless-N WPA2 network which is good as I would like to phase out the B/G WEP. Anyway, Netflix runs fairly well on it. Netflix is probably the most ubiquitous service I have access to use.

Dismayed

Internet Radio
Given that the Dash does not provide AM/FM radio built-in, this is the best app to use to listen to your local stations. Unfortunately, this looks like one heck of a unfinished app; more to the point it appears to be more proof of concept that indeed Internet Radio can be streamed and is missing the consumer oriented features that make it easy to use. In my world of development, this is would be a manual test harness the backend developer put together to prove the playing of audio streams works.

Then Internet Radio Streamer allows for user to manually add streams that can then be played. By manually adding streams, you have to know the URL of the stream and type it in completely and the content type of stream, m3u or pls. There is no pre-screened list of stations or search functionality. You can only manually add new or edit & delete existing streams. The list of added streams can’t even be reordered or categorized.

OK, so to use the app I need to use my computer to go find stream URLs and add them to the Dash. Not too big a deal since the device does allow adding and configuration of apps on the web site. I can fire up Safari, find the streams and copy/paste to the Dash configuration site. Does this app allow for configuration on the web site? No. So now you have to have computer within eyes view of Dash to enter the stream URL.

I don’t go looking for audio streaming URLs everyday, so didn’t know where to go. My first instinct was to use put in radio stations into search box, “knx 1070 audio stream url”. This had success in getting me to pages that would play the stream online, but made getting the URL difficult. I used Safari Web Inspector to find the URL. One down, many to go.

Eventually, I decided to go a different route. I eventually found some aggregator sites that provided better access to the URL. The main site I used for most stations I eventually added was vTuner. VTuner provides an aggregation list of audio streams and an API to access the list. Their API has been added to many devices, just not the Sony Dash (yet?). It seems the Internet Radio Streamer app can be made vastly more user friendly.

Responsiveness
The thing is slow to respond to inputs. I hit a button on screen and then need to wait several seconds most of the time until I can tell whether or not it recognized the input. I understand the device is intended to be inexpensive, which probably limits what strength processor can be purchased, but wow.

Parting thoughts

While I have explored some apps, I haven’t had an opportunity to fully find the ones that fully interest me. I like the Dash and can’t wait to see what updates come in the future to make it more useful.