Wednesday, May 9, 2012

About Japan

I've been wanting to post about Japan for awhile.

I was there about 2 years ago for work. It was one of the toughest times of my career so far. Away from family. Longest hours worked ever. The most a client was ever disappointed in my career thus far. I hated it.

But, I loved it too. It was heroic. Likely the highlight of my career verses the toughest. How often does a twenty something nobody from Cleveland get sent overseas for a project?

We came in on top from a delivery standpoint. I also got to meet some incredible people and experience an incredible culture. The level of respect the Japanese people have for day to day things is incredible. I ate great food.

I could write a book about my experiences, and I've tried to sum it up in some posts, and them scrapped it, so I will keep it simple. Imagine thinking about writing up a story that can fill a book 100 pages long for 2 years, and in the end leaving at this.

Looking back on the code I wrote, it's been a point of reference for many things I've done after. We raised the bar on the project, and even two years later, I encounter clients trying to achieve what someone else had already done.

I've had some photos up for awhile, you can see them here: http://www.flickr.com/photos/tepietrondi/tags/japan/.

That is it, what more can I say? Questions about traveling to Japan? Just go.

I don't want to hear it

I love strategy sessions.

"We don't have X so we can't do Y", says someone. "X is missing and it causes Y", says another.

I don't want to hear it.

You make the rules. You set the priority. If you don't have something that you want, it is by choice. Don't make noises of disgust during meetings or laugh at how pathetic something is on your site. Get it fixed, make noise to get it done, or shut up.

P.S. I love my job!

Out of the box

I am a big fan of the food network. I like the shows where there are challenges between chefs with time limits. I was watching Chopped the other night, and a chef described their cooking as "out of the box". In this context, the chef was saying that his cooking was different then standards and what everyone else might think. Out of the box means creative, inspired, or other words you'd use when describing an outcome of developing art.

In software, "out of the box" means "off the shelf", "as is", "vanilla", "standard", etc. Everyone that gets this software and uses it as they got it, gets the same thing as everyone else.

What's interesting is that sometimes out of the box is desirable and not desirable, but never have I heard anyone say on the cooking channel that they prefer an "off the shelf" chef making "standard" food.

Out of the box software and maintaining the "out of the box"ness is good for upgrades, maintenance, feature additions, etc.  But being "standard" means someone else out there, or specifically, your competitors, are using the same "box", and then what edge do you have?

If you are asking "what have your other customers done?", I think you are on the wrong track. More time should be spent on "what can we do differently?".

Saturday, April 28, 2012

Google App Engine urlfetch and twitter streaming

I spent about two days and finally came to realize the Google App Engine doesn't work with the Twitter streaming api via the urlfetch. I didn't want to do anything specific (yet), I just wanted to see the streaming api run on the engine and then figure out what to do next.

Regardless, it doesn't work. I even tried to change the implementation in Tweepy, but that wasn't the problem. Tweepy uses the httplib, so I thought I'd try to switch it to the urllib2 implementation. Same problem, when you open the connection to the streaming URL end point the response object never comes back to read on the App Engine. Run it on the command line, everything flys through the stream. You can see my forked version here of my attempt to change the Tweepy streaming implementation: streaming.py. Or you can see the commit differences as well for streaming.py.

Another trick I learned in this is adding the basic auth to the Tweepy streaming implementation. The documentation and examples are confusing for the first time around. I was trying to pass the oAuth to the streaming implementation, but it takes a basic http auth. See the gist I referenced: Twitter Streaming API sample using the filter stream. Only difference is I used the "sample" and they used the "filter" streams from twitter.


I am surprised with Google and why this implementation works the way it does. When I think "scale" for processing a Twitter feed, wouldn't Google want people to think of them as a platform to implement a solution for? Or any streaming HTTP endpoint for that matter, regardless of it being Twitter or something else (a log file for example). Hopefully they will improve this soon.

Some other resources:

Friday, April 20, 2012

The worst way to have access to logs

Do you have to access logs for your job? What the worst scenario you can imagine to make it hard to get to those logs and work through them? Let's try this:
  • Logs are on a windows box
  • To get to the log machine, you have to remote desktop to box A
  • But the logs aren't on box A, the logs are on a share via box B,C and D
  • You can't RDP to box B, C and D, so you have to map the drives on box A
  • Why do I need to map the drive? Because it's Windows and there are no Unix tools available
  • Cygwin is on box A not the others, so when I map the drive, I can access them via Cygwin
  • Usually just mapping the drive sucks for speed, because the I/O is via the mapped network connection, so this forces me to copy the logs to from B,C and D to A
  • Once moved, I can now grep and open logs in vi for search
Why is this bad? When an error occurs that we want to catch before the logs roll in live production, we need to get to the logs quickly and move them. This process above takes like 15 minutes even to begin looking at the logs. 

Why aren't the logs archived? They are, but I dont have access to that box either.

To top all this off, lets add a site to site VPN tunnel to slow the connection down even more.

How can this be better?

  • Give me access directly to the box with the logs
  • Install SSH on the box so I dont need a windowing system to get on (I don't need a host Unix system or variant, just some SSH action)
  • Aggregate the logs to a single location real time (using Apache Flume as an example)
  • Setup some log monitoring that automatically detects errors, fires off alerts with the relevant log details
  • Etcetera, etcetera, etcetera 

Friday, April 6, 2012

My MacBook doesn't have a touch screen

I was showing an elder some stuff on the web the other day, we were doing some research together. I don't think they ever used a Mac before, so everything was new, but we were just using the browser.

I had to show them how to scroll on the touch pad with the two finder swipe. They figured out the click fine even though there are no left and right click buttons.

In the midst of talking, the person touched the screen to click a link on a web site. I smiled, and clicked it for them using the touch pad.

Interesting what tablets and mobile devices are doing to people's expectations, however I dont think this person has used one of those either. Maybe just saw someone use it once and assumed my laptop worked the same.

See also: A Brief Rant on the Future of Interaction Design

Monday, April 2, 2012

How to break javascript compression using eval

Working on some javascript compression (or minification) for a site. Given this javascript:
function callingFunction(arrayVariable){
  var jsEval = 'globalFunction("String parameter " + arrayVariable["lookup"]);';
  return jsEval;
}

// non-global
var arrayVariable = [....];
var actEval = callingFunction(arrayVariable);
eval(actEval);
When the compression is ran, the variable in the function definition for "callingFunction" will be compressed, call it "a" from "arrayVariable" because it is not global. So now when "callingFunction" executes, and generates the "jsEval" to return a string with the non-compressed variable name. The execution of the eval will result in an exception of "arrayVariable" not being defined.

This is how you break javascript compression, or in other words, write javascript badly for compression. The right way? Don't use eval, or:
var jsEval = 'globalFunction("String parameter " + ' + arrayVariable["lookup"] +');';
Code might have an error, but you get the idea right? "arrayVariable["lookup"]" is being evaluated during the setting of variable "jsEval" verses the outer "eval(actEval)" execution.

See the following via the google closure compiler tutorial:
"Compilation with SIMPLE_OPTIMIZATIONS always preserves the functionality of syntactically valid JavaScript, provided that the code does not access local variables using string names (with, for example, eval() statements)."
And again Broken References between Compiled and Uncompiled Code:
'Keep in mind that "uncompiled code" includes any code passed to the eval() function as a string. Closure Compiler never alters string literals in code, so Closure Compiler does not change strings passed to eval() statements.'
You can also try the YUI Compressor.

Getting revision history from cvs

I had the worst time a week ago trying to get the history from cvs at a top level directory using eclipse. However I there is no option to right click a project directory, or any directory, and show the history for that item.

To get all my commits I came up with the following from reading around forums and reference sites:
cvs log -N -S -r -w > branch.log
I have no idea what all the options mean, but the "-w" are my commits and "-r" is the branch I am getting the log history on. An important point here is you will note I am feeding standard out to a file because this command prints all kinds of warning information I don't care about to the standard error stream.

The file will show something like this as a result:
RCS file: %FULL PATH FILE NAME IN REPO%,v
Working file: %FULL PATH FILE NAME ON FILE SYSTEM%
head: 1.70
branch:
locks: strict
access list:
keyword substitution: kv
total revisions: 77; selected revisions: 1
description:
----------------------------
revision 1.67.4.1
date: 2012-03-21 10:18:43 -0400;  author: %USERNAME%;  state: Exp;  lines: +4 -2;  commitid: hhCEg6Vtn6C46LXv;
%COMMIT COMMENT HERE%
=============================================================================
The key to finding my changes per ticket is in my "%COMMIT COMMENT HERE%" portion where I prefix my comments with the ticket as a standard.

This literally took me like 4 hours to put together due to my desperate attempt to find a solution working within Eclipse. Once I gave up on that, I started looking at a command line reference.

See also:

Sunday, March 25, 2012

Since forever

I was at the bank recently. I was asked for my employment due to the nature of my visit. I was asked how long I've been in that line of work. "Since forever" was my response. The women working with me laughed and looked for my date of birth. "I've been doing banking since the year you were born, you've been working total since 2005!".

It's felt like forever. I've been doing computer / web work way before I've had an official job out of college in 2005. Since like 13 I've been developing. Classes in high school. Classes in college. Side projects. Then finally official work.

It has been forever, and I have a long way to go till retirement. However, when I retire, I can't image at this point doing anything else in my free time alone. That might change as I get old, and prefer free time spent on a beach, boat, or with my family. But for now, I'll will be doing this (being development, computing, the web, etc) forever.

Sunday, March 4, 2012

Working from home, a year later

It has been almost a year now since I've been working from home. A few times during the year, I've had ideas on writing up recent changes to my life, but I've chosen to wait until now to really write out the changes. The following is a simple break down for anyone interested in what it is like working from home, or wanting to compare their home experience with mine.

Time online

Working from home causes me to be online more that I thought I would be. There is less downtime to and from work. Less interruptions from in person visitors, shorter lunch breaks, more productivity multitasking during calls. Less team collaboration, more heads down focus, more time to think about my work.

Attention span during meetings

For whatever reason, during calls, I cannot pay attention any longer due to multitasking. Unless directly involved, I usually have no idea whats going on and have to ask for questions to be repeated. Personally, I've come to think that meetings in general are not productive unless solving a problem and the invitees have a direct stake in the decisions being made. Otherwise, write it down.

Range of motion / health

I move a lot less than if I were going into work. I dont walk up and down steps to get to certain floors, I dont walk to my car in the parking lot. I sit way more than I should. I try to stand while I work and work out, but in general I am lazy, so I dont work out or stand. I get out of my seat a lot. This was a normal habit while I was in an office. I would always get up, walk around, get another coffee. I do the same at home. I go down into my kitchen, I walk outside and look around. You can't sit all day, or you will die.

For my health, I am definitely less active in the winter. I used to walk couple times a week on the park trails and lift weights in the warmer months (more sunlight). Now I sometimes remember to work out as mentioned. I change chairs that I sit in every once and awhile to give my body something new. I eat house food for lunch rather than eating out. I do miss getting lunch with coworkers.

I've cut back my coffee intake while at home. One might think it's easier to drink more. Well coffee was free at my last job and I was way overboard. I've cut back to half.

I wear my glasses over contact lenses giving my eyes a break.

Driving abilities

The few times I'd had to drive during rush hour have allowed me to reflect on my driving abilities. I am less confident and would rather drive during less busy times. The stress isn't worth it.

Costs / Money

You might think its cheaper working from home. I spend less money on gas for my car, less miles on my car so less maintenance, less miles means less insurance. I also have to heat and cool my home to normal temperatures now since I am home. In the summer when nobody was home, the AC would jump to like 78 and then down to 73 later when I got back. Same with heat, used to be like 60 when I was gone, 68 when I got back. Now the AC runs steady, so does the heat at livable temperatures. I also use my own water and electricity all day. So it balances out I think.

I don't use my cell phone much anymore. I use my Google voice number for calls online. Sometimes I use my land line when the voice number rings for work. Otherwise, conference calls always have computer headset capabilities. I bought a simple wired headset for this, the trick is that muting and un-muting calls is not as easy as a button click on a physical desk phone.

For whatever reason, I have a laundry and dish problem I need to solve. It's just me and my wife and the dishwasher runs every other day and we do laundry twice a week. I have no idea why. I have "house clothes" now, which are outfits that are suitable only for home, and not appropriate for public. I wear these same clohes week after week.

Conveniences

Yes I sleep in now. I used to wake up as I would a normal job, shower, get dressed and go to work. That took about 9 months to go away. Something with the new year caused me to stop setting my alarm to wake up and its been a nice change just waking up naturally. Yes I can leave the house whenever, work where ever and whenever. I run to the store if I need to for breaks, no issues taking off for the doctors. I can make up my time whenever as long as it's in when ready, so being online from 9 to 5 doesn't really matter. I cut the grass during the middle of the day normally. Start dinner around 4:30 and go back to work when done.

On Fridays, I have a beer while working towards the end of the day. This is usually when I am entering my time for the week and tying up loose ends.

Side activities

Yes I have more time to blog, tweet, and get involved in side projects to keep me busy. I can even do this during the day. I thought I would be inspired to startup some major idea or little side company, but I've been enjoying the time to myself that I get back with my new job (it's a steady 40 usually).

Distractions

"Don't you get distracted by family or your wife?" Some, but it's good to break up the day and get out of the chair. Normally, I wouldn't be home and couldn't help someone move or carry something. You might think this is a terrible distraction and then get angry because "you are at work". I suggest just going with it, you are home, you won't be able to avoid this.

I try to go out and work somewhere else at least once a week (other than my house). I have a few buddies that we meet up at a coffee shop and work together. If you don't have buddies, or nobody is available, just go somewhere. Or go to your parent's house.

Internet and bandwidth

One downside with working virtual is sharing large files. We usually upload to Amazon S3 or put USB drives in the mail. My home internet is nothing great so sometimes file transfers set me back. I usually have to wait until night or over the weekend to resume transfers. I've wanted to update my internet package, while at the same time reducing my bill, so it hasn't really happened yet.

Summary

In summary, the number one perk is freedom (convenience), the number one downside is relying on my internet upload bandwidth.

Some advice, don't take a job just to work from home, it not everything, it's a perk. I enjoy what I do so working from home makes it better. You can't make a bad job better just because you aren't in the office.

If it matters, I live in the northeast region of Ohio. I could work anywhere, but my family is here, so I haven't considered moving anywhere warmer yet. Maybe one day, but I like where I live.



Wednesday, February 29, 2012

Short story using the noun project symbols

If you haven't heard of the noun project, check it out, it offers public symbols for demonstration of concepts and ideas through images. Visualization can be very powerful for the portrayal of ideas and concepts verses language alone. This is why visualization as a whole is making a big splash on the scene, and not just symbols.

Regardless, for the longest time, I've had an idea for a short story, but I have no artistic skill for drawings, so I put together the story using the noun project symbols. I was hoping it might appeal to children as the idea was inspired from my family, specifically my mom and my sister's kids.

Regardless, I wanted to share if for free since I don't know anything about publishing or making money. Let me know what you think and if you like it.





tepietrondi/The man with no bed. @ GitHub

The next step is to have my nieces and nephew maybe jazz up the pages with some kid color slop, but I think the black and white looks nice.

Sunday, February 26, 2012

Developer state of mind

Let me spell it out for you, put it in words...

If you are a developer, admin, or some other technical computer science oriented person in your day to day, and you walk into a room or join a call with others of the same background, there is no room for feeling intimidated or being less superior. You better not care about years of education, years of experience. Know you are the man and everyone has to get on your level. There is no other way to cary yourself in your own mind in this field. You better feel like you are Neo at the end of the Matrix when he flexes in the hallway and the room bends around him. You better feel like the guy who needs the ball at the end of the game to make it happen.

There is a difference between acting out this arrogance and thinking it. I wouldn't recommend acting this out prematurely, we all know what happened to Anakin Skywalker. Not until you are a Jedi will abilities show and be know, but until then, you better believe in yourself to get there. If not, you're getting get walked on, left behind, and not thought of when the heroic moments are needed to bring back balance to the force.

Listen to some hip hop if you need to, these guys are all the man and nobody has ever heard of them. It's the attitude and self confidence that is needed, not saying be an asshole, just feel good about yourself.

One more thing, if you are not me, your code sucks.

Sunday, February 12, 2012

How to break the internet

If you follow me on twitter, you might have seen this post:



Basically my wife broke the internet and I wanted to explain how.

She does her own site management for a small business via a microsoft service (office live small business or something). The service is transferring to something else and we took the opportunity to get off the platform due to personal preference. Honestly, for someone who knows nothing about making a site from a code perspective, I thought the microsoft product was just fine, so no knocks against them.

So in the transition, we demoed a few other similar services and chose intuit. Same style of making a site, some things better, others not. No big detail, she made the new site in the editor, added some pages, done.

Now the domain transfer...

For everyday people, this isn't so easy as making a web site without code. There are transfer codes, locking, name servers, etc. So I told her I would do this with her so we could maintain the same domain she had for the old site and reuse it for the new site. What happens? She closes the account with the old domain. The domain is locked and the contact email address on the whois is also deleted when she canceled the microsoft service.

So, what does she do? She buys a new domain and then updates the company facebook page and sends out an email to the mailing list. No big deal right? Everyone will get the new link and all is well.

Not in my eyes. That old domain is everywhere, its on business cards, menus, likely bookmarked and most of all, indexed in all major search engines and local map pages as the company URL. This cancellation effectively broke the internet. The URL is everything, if you've read my handwritten post, you'll see some links and where I explain responsibility of ownership of a URL. It's everything. If anything else, it needs to redirect to the new domain rather then just being a dead 404:

HTTP/1.1 404 
Connection: close
Date: Sun, 12 Feb 2012 15:13:21 GMT
Server: Microsoft-IIS/6.0
MicrosoftSharePointTeamServices: 12.0.0.6043
X-DIP:202
MS-Author-Via: MS-FP/4.0,DAV
MicrosoftOfficeWebServer: 5.0_Collab
X-UA-Compatible: IE=EmulateIE7
X-Powered-By: ASP.NET

So I am going through the labor now of submitting documentation to the registar of updated contact information for the domain owner to get back access and point the old URL to the new site at least for a year. For the company name which is somewhat common, it is first in google for that term based on organic results. Unless recovered the internet has a hole in it. The library of congress, google and whoever else downloads the internet for indexing won't know what happened. Machines won't understand.


Tuesday, February 7, 2012

Recipe for JSP / JSTL cross site scripting vulnerabilities

I put this together to grep through JSPs looking for possible cross site scripting vulnerabilities.
grep -R \$\{*.\.*.\} --exclude="*\.svn*" --include=*.jsp* * | grep -v c\\:out | grep -v c\\:set | grep -v c\\:when | grep -v c\\:if | grep -v c\\:forEach | grep -v c\\:param | grep -v fmt\\: | grep -v c\\:import | grep -v jspStoreDir | grep -v pageContext | grep -v svn | less 
It's not perfect, but it helped me fine some potentials outside of the security scans. The difference being I have access to the code, and the security scan doesn't. This was used on a Websphere Commerce implementation specific to the Stores directory.

You'll have to page through the results using your own experiences to actually locate the issue. This just helped me filter out some items. For example:
<input type='hidden' name='productId' value='${WCParam.productId}' />
This item is directly output to the page verses using the "c:out".

Monday, February 6, 2012

More examples of terrible recruitment practices

Couple more terrible approached to recruitment...
  • Asking me for information on others you failed to connect with. No, I will not give away private information of my network. 
  • Guessing my work email address and sending me opportunities. Clearly I don't list my current employeer email on my resume and profile for a reason, it's not my personal email box, it's work. Why are you guessing my email given my first and last name and expecting me to respond? My actual email is very easy to find. 

Share on Twitter