Sunday, January 5, 2014

Camel Route autoStartup via Property Placeholder

It might not be obvious in the Camel autoStartup documentation, but this configuration can be managed via a property placeholder. Such that the route definition would look like the following:

<route autoStartup="{{route.feed.autostart}}">

Why would you need to do this? In my use, the property placeholder variables are mostly environment specific, allowing deployment of different ".cfg" files per environment (such as development verses production). Given a situation where your development environment consist of more than one environment, such as a QA environment, but the integration destination of the route data only has one, then you wouldn't want to execute the route in both the development and QA environments, but only one. Then, in your property placeholder ".cfg" files, you can map the autoStartup flag to be environment specific.

Saturday, January 4, 2014

ActiveMQ Connection Factory Timeout for Failover

I recently encountered an issue within our Apache ServiceMix environment such that if all the nodes are down that are participating in an ActiveMQ failover connection, without a timeout setting in place on the connection factory, ActiveMQ will wait forever for a node to become available.

This sounds reasonable, but within our usage, other processes were beings blocked and rolled back which were unrelated to the ActiveMQ integration messages. At a high level, the sending system would batch send messages to integration points A and B which are of different types. Type A might be a file system write, while B is an ActiveMQ message send.

If the message ordering was as mentioned above, all would have been fine if ActiveMQ failover nodes were all down. The file for message integration A would send, B would wait forever to send, and nothing is after it, so no harm done. Problem is, the process for the batch send of the mixed messages types will run again, and thread for sending integration B, or ActiveMQ is still waiting. The backend of the messaging system is a simple database table, which stores the messages for sends, and clears them after. So now a second process thread attempts to send message B integration again, but rolls back given that its locked in the initial attempt, and we begin to see a symptom of the original problem, the blocking.




The original failover string in the ActiveMQ connection factory looked like the following:
failover:(tcp://10.1.1.1:61616,tcp://10.1.1.2:61616)?randomize=false
Notice no timeout, resulting in the blocking forever. In our case mentioned above, we don't want the connection factory to wait forever, we want it to timeout, so the next batch message sending process can attempt the send. Referencing the following ActiveMQ documentation:

http://activemq.apache.org/failover-transport-reference.html

After adding the additional parameter to the failover string, that being "&timeout=<some number>". The result would cause the connection attempt to stop looking for an ActiveMQ node to take the messages. Once added, the sending thread fails on the ActiveMQ message rather than blocking.

Also note, the inclusion of the JMS header "JMSExpiration" had no affect on within the message since the issue was within the connection establishment, not the actual sending of the message.

Obviously you might think, "why would all the nodes in the ActiveMQ failover be down?". The answer is, they were down.

Friday, January 3, 2014

Resolving Apache ServiceMix JDBC JAR Classloading

This past year I've spent some time using Apache ServiceMix for service bus integration. One of the main issues I had was integration with database systems and using their provided JDBC drivers when defining the connection bean within Apache Camel. Many of the documented approached and user group threads did not work for me in environment. The main approaches being wrapping the JDBC JARs as OSGi bundles and / or installing the JDBC libraries as a maven repository so the dependencies are loaded properly in the ServiceMix container.

To resolve the issue in my case, I had to deploy the given JDBC JARs into the JRE library extensions (lib/ext) directory to allow the classloader to locate the libraries correctly. Once done however, I was not complete in the integration as the bean was not being detected as available within the routes in Camel as documented in the JDBC documentation. The connection was not available for passing data directly within the route. As a work around for this issue, I was forced to pass the JDBC bean reference in Camel into the depending bean as a reference. For example, if JDBC bean "model" was defined, and my logic bean as "controller", the "controller" bean definition in Camel looks like:

<bean class="com.controller.Service" id="controller">
<property name="dataSource" ref="model">
</property></bean>

Within the "Service" object, would be getters and setters for the dataSource field as type "org.springframework.jdbc.datasource.DriverManagerDataSource".

Oh, and one more thing. The deployed JDBC libraries in the lib/ext directory would not be loaded given security trust exceptions, similar to this thread reports. As mentioned in the linked article, the fix was to deploy sunjce_provider.jar into the lib/ext as well. One implemented, the container was able properly load the third party JAR files.

All mentioned was done with Microsoft SQL Server and IBM DB2.

See Also: https://github.com/cschneider/Karaf-Tutorial

Thursday, May 9, 2013

Memory verses bandwidth

For work I run VMs that contain toolkits for development on different clients. I used to run these locally. My limits for working on more than one client was limited by memory. At about 8 gigs of memory, you might be able to run two smaller VMs, but it got slow, so I never bothered.

Lately, I connect remotely to hosted toolkits. I can work on two clients now given my bandwidth. It's not perfect, I could use more, but it works. I could do three if I had a local toolkit and two remote toolkits (I do this for internal company work on side projects). Same screen paint issues regardless if remotely connected to one verses the other. Now my local memory is free on my machine, but I use it for browsers, a shell and twitter mostly.

One last thought, if you want to know how your application performs over a site to site VPN via remote desktop with average bandwidth, contact me for some test. I really hate how Firefox repaints when you refocus to it. I really hate how Eclipse flys out all those bubbles for suggestions when you hover over something.

Monday, April 29, 2013

The bench

I don't know how common the concept of "the bench" is in consulting, but for my current and last positions, each firm has had "a bench". As in baseball, those not actively playing are on the bench, then when a project opening comes up for filling, you are put into play.

You can be on the bench for a number of reasons, but mainly the bench ends up having the weaker individuals or greener resources. The members are coming up, learning, or just suck.

This is fine, but what confuses me always is the type of work given to the bench. It's usually the work I want. Like, "go figure this out" or "automate that". These are huge opportunities, some rabbit hole the company has that could be fantastic, they give it to someone who might not be ready for it. Sure it might fail too, but these little projects are always the wild success stories we live for in technology and I want to work on them.

Most other companies give this proper names like "the lab" and the strongest most experienced resources are on this type of work giving it the proper attention. Why it has been downgraded in my current and past firms, I will never understand.

Saturday, January 26, 2013

Forced direct learning

I think I know what's wrong with kids ... forced direct learning. Everything is about learning. Letters and numbers on toys. Toys are colored. Toys have shapes. Toys play music and sing along stories. Books have buttons and noises. "Baby can read" movies. Mickey talks to kids and waits for them to respond to how many of something there are on the screen (it is the most awkward time used during a tv show I've ever seen, they should use the dead time for ad placement).

What happened to kids just having basic cool shit? Wood blocks, no color. Some balls. Bikes. Action figures. Dolls. A yo-yo. None of that has anything to do with direct learning. Bang stuff around, throw it, build. Then you go to school and learn. Learn from your parents. Learn indirectly from the interaction of things around you. What happened to cool ass cartoons with stories that taught lessons through actions rather then having counting and color games, plus dead air time with funny cartoons looking back at you with extremely large eyes?

Kids figure out they are being taught, forced, then they don't care. They realize they could have had remote control cars verses the color learning truck. Then they put aside learning and beg for cool shit you never gave them.

Give your kids some basic cool shit, be a good parent, and then they might care about direct learning in school. In 10 years they aren't going to accredit their intelligence to a jump start they got with that alphabet bear you gave them in place of reading with them.

You know who likely had some cool toys when they were a kid? Einstein ... from Wikipedia:
His father once showed him a pocket compass; Einstein realized that there must be something causing the needle to move, despite the apparent "empty space".
A damn compass! I bet his dad never explained why and how it worked to him. This is what I am talking about.

Executing sftp commands via ant

I've been working on porting some korn shell scripts to apache ant lately. In the rewrite, I had to implement some sftp steps. Some were straight forward, get file. Others, I had to remove files via the sftp connection. The ant "scp" task doesn't provide this. So, I had to basically mimic the shell script implementation. Here is a sftp directory list:


I learned two things here. First, how to pass commands in general via sftp, which I never had to do. Second, how to do it via ant, meaning not doing it in ant and calling to the shell via ant.

Reference:
See also:

Sunday, November 25, 2012

Cleaning your workspace

When I worked in an office, I used to like taking the time to clean my workspace on the company dime.  Get the cleaning spray, wipe down the desk, chair, mouse and laptop. Then reorganize the space on the desk, getting rid of the nonsense accumulation that adds up over time.

Now that I work from home, it's normal house duty rather than a break while on the clock. Usually happens after hours or on the weekend.

My advice, or point? Take a break if you are in an office and enjoy cleaning your space.

I guess there is actually an official day for this as well: CLEAN OFF YOUR DESK DAY


Friday, November 23, 2012

Tis' the season for kids and tablets

Not sure why, but I think this holiday season is going to bring some changes to the tablet market. I keep hearing who is getting a table for themselves or for their kids. I don't (and won't) have the data, but I expect we see some break through numbers with browsing and access to the web via tablets in an age bracket we've never seen before.

I am realizing it's no longer a joke for kids to have devices, it's a real thing that won't be stopped. I tell stories like, "at that age I was swinging sticks in the yard", while today kids are online.

The landscape has officially change. While I cheap out and hang onto my generation one iPod and iPad, kids at age 5 will have faster and lighter experiences then me. All touch interfaces. Geolocation. Cameras for pictures and video.

All with terrible parental controls. All with terrible examples of usage by parents pouring information onto the web that they'll one day want to take back, but can't.

It will be interesting, and I am not even talking phones, just wireless connectivity. Voice communication is really irrelevant in my opinion. There is more to worry about with photos, video and texting getting online verses phone to phone.

Happy Black Friday.

See Also:


Friday, November 16, 2012

Morning orange juice and meetings

I've been cutting it close for morning calls lately. Eight AM is early when you work from home. It's my natural wake up time. But when I have calls at 8 AM, that means I need to get out of bet earlier to hit the shower and make the call on time.

So, I've been getting up at about 7:45 and then get on the phone by 8:02 (usually still late). Since it's still wake up time, I've been taking my calls at the kitchen table without the laptop. I pour my orange juice, sit, listen and look out the window.

My listening abilities while in front of a screen (tv, ipad, laptop, phone, etc) diminish greatly. I only ever focus when something odd is said or my name comes up. So being at the table with more focus has been nice.

With the change, I've also not rushed into the day walking right to the laptop. I complete the orange juice intake, and then make coffee while all on the call. Speaker phone of course. End the call, then move upstairs into my office.

Taking calls in the car is similar (no screen) however I usually am focused on not killing myself and other drives while participating.


Thursday, November 15, 2012

db2 load Over JDBC Connection

Yesterday I needed to run the db2 load over a JDBC connection. Given the "connection", I was not local to the server. With the help of a colleague, I found this page:

LOAD command using the ADMIN_CMD procedure

The resulting call in ANT is:


Note that the load file needs to be on the remote server, verses being called locally. So you'll need a step to move your load file to the remote machine.

Saturday, September 22, 2012

robots.txt and sitemap.xml on Google App Engine

I added a robots.txt and sitemap.xml to my Google App Engine site (http://twix-mit.appspot.com). Due to this change, I also added "when" pages. From when pages, I added better HTTP caching given a cron cycle.

robots.txt

Source code:
Handler:
Runtime: http://twix-mit.appspot.com/robots.txt

sitemap.xml

Source code:
Handler:
Runtime: http://twix-mit.appspot.com/sitemap.xml

"when" pages

You might note in the sitemap, all the pages are driven by date. I've used some of the nice time delta modules in python.

Source code:
This allows getting the current time, doing delas to tomorrow, yesterday, etc.

"when" caching

Per the "when" stuff, the http response headers are now timed given the cron cycle for the news digestion. The memcache time is also driven by this calculation as well.

Source code:




Saturday, August 11, 2012

One hundred percent

Way back, I used to work at a service department for the city that I lived in. I had normally worked summers and gone back to school. My last year, the head of the department let me stay on for leaf season in the fall and work the weekend (Saturday). My responsibilities were mainly to keep up the lawn, so cutting grass, blowing glass and weed whacking. For everyone else, it was leaf season, meaning they drove the trucks around to suck up the huge leaf piles on everyone's tree lawns.

I didn't help because I wasn't on for leaves, I was on for cutting grass, and so I got paid the grass rate which was less than the rate for leaves per hour. Doing leaves is hard work, you have to mostly rake the leaves to the suction tube rather than moving the tube. In the Fall in Cleveland, it's usually wet as well, making the leaves heavier.

The one day, the crew asked for my help as they were behind and needed to catch up. They were salary since they were full time, I was part time, and so hourly. My response to them was, "no". I got sour looks from most everyone, but one guy spoke up, "no way, 100%, he isn't helping".

Now whenever someone makes mention of "one hundred percent" I usually think back to this moment. Was I right? I think so. Regardless, someone else thought so too.

Friday, August 3, 2012

What do you think Nathan?

Me: What do you think Nathan?
Nathan: That'd be annoying to walk around.

Few days ago I was outside talking with my wife about putting some sort of outside containment system in place for our garbage cans. They are currently in the garage because the homeowners association says they can't be in plain view outside in the front of the house. However, they do allow them outside if they are fenced in (the containment system).

So we are talking about how this would look, how that would look, where it should go, etc. Then my neighbor got involved, older woman, a grandma. Then my neighbor introduces her grandson, Nathan.

Nathan blows my mind by offering the comment above. I never thought about how it would impact my habits of walking to my backyard, or being in the way. I was concerned with looks. Nathan has to be like 8, I am older than that and educated, yet his comment stopped us dead from putting in the structure.

The point? Kids are interesting.

Thursday, July 19, 2012

Do you know why the people on Jeopardy are smart?

My grandpa used to say to me, "do you know why the people on Jeopardy are smart?". His answer, "because they read books".

He'd tell me this once a week because he never remembered he already told me.

I don't read many books. I want to, but never do. When I do read books, I read slow. Few pages at a time, then I close the book to think about what I read. When I read fast, I always day dream away out of the book about something else rather then the material. I never read good in school either, or at least when it came to literature, or novels. I was always behind, missing the story line and never knew anything about the characters.

When required to learn a second language in college to graduate, I did terrible. I barely can grasp my own native language let alone another.

In conclusion, I don't think I am very smart, but my dad says I wouldn't have my job if I wasn't (he didn't go to college nor does he understand computers). My wife says that I am smart at what I do for a living, but otherwise lack most common sense. My friend is a teacher and believes he is degrees smarter than me, but makes a lot less money than me. If this were true, wouldn't he know how to make more money and apply his intelligence there?

I think there are that many more people less smart than me that makes me appear smart, but again, I am not that smart at all.

Share on Twitter