Families of Programmers

throughout my life as a software creator and destroyer, i have come across a lot of programmers. i found that there are different kinds of programmers are there are with all things. this is just my opinion of course, but i want to show that not all programmers are alike and some are as different as a mechanic to a lawyer.

The OS Developer
also known as the "kernel hacker", these people are the epitome of software engineers. they created UNIX, worked at Xerox Park, and built the internet. these guys created CS theory. They understand all the intricacies of how threads and processes work, and can move bits around with their eyes closed. from a pure technical sense, these people know their shit. there's a problem though. these guys arent really going to change the world from a perspective that anyone can see. now dont get me wrong, they do really awesome things that other people will use forever. they are so far away from the user that they dont need to ever think about them. i dont know which is causing what, but there seems to be a direct correlation between an OS developer's inability to create compelling software and their technical aptitude. if they were the only people writing software, no one would have ever heard of the computer, the internet, or the world wide web. it would remain a geek's plaything that only a small subset of the population would know about.

The Hacker
this isnt the evil hackers you hear about on tv, but they also arent the old bearded MIT hackers you read about in books either. my definition falls somewhere in between. they like to tinker with stuff, reverse engineer binaries, and make things work in an unintended way. now, these may sound like the hackers of the golden days, but unfortunately, most of their efforts are spend in OS exploitation, rootkits, botnets, defacing websites, etc. there are definitely hackers doing more worthwhile things, but my point is that their energy is focused on security which sets them apart from the oldschool hackers.

hackers are bad coders, or rather, they are not required to be good coders to do what they do. they mostly use glue code to piece together tools they find on the internet. even super elite hackers will only write small chunks of C or asm code for some exploit. the code hackers produce rarely has to be reused. they never have to make robust software that scales and serve millions of people. they just write little pieces of code to do cool shit. these guys dont really know CS theory, but then again, they dont need to. if they are really hardcore, they will know a lot about networking and OS theory, but that's about it. dont ever tell these guys they dont know CS theory, because most of these people will think they are the shit at everything. if you spend a lot of time gaining access to servers that contain a ton of sensitive information, you tend to get a big echo, even if what you're doing isnt technically advanced, it does give you a lot of power.

the good guys will write hacking tools and come up with exploits for the rest of the world. they notify companies when their systems are vulnerable and release advisories to keep the world alert. then there are those who keep that type of information for themselves and comply with a strict policy of non-disclosure.

The IT Professional
i almost want to exclude these guys from the list because i dont think they're programmers at all. they know how to install, setup, and maintain preexisting software. usually, these are the geeks who cant code. they play around with linux all the time and learn how to setup stuff like sendmail, mysql, apache, ldap, etc. these people usually end up programming little tools that no one can or will want to ever reuse. it just helps them. for one reason or another, they dont have the desire or skill to hack anything, so they just sit there and setup computers and user accounts. i dont mean to be down on IT people. they are good people and there's nothing wrong with what they're doing. i'm just saying they're not programmers.

The Web Developer
the web developer has emerged only recently. i cant really make any assumptions about the technical background of these guys because they come from far and wide. on the one hand, you have these hardcore distributed systems people who need to make web applications for whatever they're doing because the web has become so pervasive. on the other hand, you have the weenies who just do php and html/css to make pretty websites that either show dynamic or some other web 2.0ie thing. these people are more connected with the user. they usually have to do some sort of UI design. for example, you can certainly tell when a hacker makes a website versus a web developer. hacker websites are either plain, green and black (matrix), or they actually try to design it and it looks really ugly. i dont know which one is worst.

as for technical ability, i guess the 2D scale goes linearly or some sort of curve where the more they know about the backend, the uglier their frontend is, and vice versa. it totally makes sense though if you spend all your time with structure and architecture stuff, then you're going to end up thinking that way when you work on the UI. that's why abstraction is good, so you can have different people of different talents work on different aspects of your web application. these generalizations are reflected in the languages they use too. on one end, you have people who use java and C++ for most of the distributed stuff and then they try to extend that to the frontend. those frameworks are restrictive but powerful. its hard to do "cool web shit" on them, but everything is rock solid. then you have the people using php and perl who are allowed to write god awful code that is not extendable, but it gives them the flexibility to do all this awesome stuff with javascript and and html/css. some frameworks are trying to bridge the gap like wicket for java and objects for php.

with the way that the web is taking over the world, it is hard for any of us to avoid doing some web development at one point or another. i guess you just gotta choose what kind of programmer you're going to be.

The Applications Developer
these guys are closer to web developers than anything else. since this discipline is older and there is more work involved than developing web applications, there are usually two groups. they are split into application logic and application interface. the logic guys deal with the functionality of the application. i guess you would say all the heavy lifting. while their foo is not as great as the OS developers, seasoned applications developers know their stuff. they are all about data structures, networking protocols, and algorithms in general. the application layer is where you can feel directly when something is inefficient. while an inefficiency in the OS can be felt throughout, they are shared throughout all applications (mostly). it is hard to distinguish that in a particularly application. now if your application sucks, then people who use it will notice because they spend most of their time using other applications on that same OS. desktop applications are generally more complicated than web applications and most of these guys earn their paycheck by doing business software for the big boys like oracle and peoplesoft. reliability is a must.

the GUI guys i would say are also more disciplined than the web UI guys. like i said before, their applications tend to be more complicated. toolkits have been around for longer and they are also right there on the OS. they can do pretty much anything they want. if they want transparencies and things flying around, they can do that. users tend to be more sympathetic to website UIs barfing because they can blame it on the "net connection", but not so for applications. if something in the GUI doesnt work right, then heads will roll. just look at your average microsoft office application UI. it is much more complicated than any web application you can find, even the google word web application. personally, google word is enough for me, but there are some features that people use that is hard to implement on the web, such as track changes.

application developers tend to be the more straight edged version of programmers. they dont do anything cool with the internet. they dont do security, they dont hack into servers, and they dont make the headlines. because of that, they tend to be the people who arent geeks and are just programming for a living.

The Embedded Software Developer
these guys are probably some of the dullest people on the planet. according to society anyway. they are the version 2.0 of old school programmers who are constraint by memory size, cpu speed, etc. they also never get any praise or news coverage because their work is so fair removed from the public eye. dont for a second think that they dont concern you, because everytime you start our car or pay your parking ticket, you are using their handiwork. these people get giddy over some obscure method they found to calculate your change faster and with less memory. there is a whole other class of knowledge that they use on a daily basis that some of us never have to even hear of our entire lives. they deal with bitshifting things around and the like. i mean i totally love these guys to death, but they're just a whole other breed.

if i can say anything about these people, is that they lik to tinker. they were the kids who fixed your VCRs and played with ham radios. they sound a lot like old school hackers, and to a degree they are. they just ended up never getting into security or having that mischievous streak. they liked to solve puzzles and try to out-clever themselves. i would say that their patron saint was steve wozniak if the hackers didnt already have him.

these people dont care much about algorithms except for the basic ones. things that they were on usually dont scale that much to require "those" types of algorithms. they're more like computer architecture guys, adding caches here and there and prediction logic to try to squeeze out that extra 2% performance.

if you meet one and you're not already one of them, my advice is to never ask them what they've been working on. they may just tell you, in great detail.

the browser is NOT an OS

if you understand software, then you will understand why the world wide web is a piece of shit. its really not the inventor's fault, but rather its users. the www started out as a way to visually provide information. gopher was cool, but people wanted things to look pretty. i have no problem with pretty, in fact, i love pretty. i own apple products because they are sexy.

i believe, the problem started when dynamic pages started being popular. it allowed for websites to do things such as e-commerce, web apps, and all this other shit that people want on their browser. and shit, i want that stuff too, but the closest medium to make that happen was the web. the problem is, it wasnt meant for that. the browser wasnt suppose to allow you to run arbitrary code you retrieve from a server. i mean, what the shit is that? it does because people can then make money. again with the money. it was also easy. everyone had a browser so they didnt have to install anything. the future looked bright.

people who make decisions about technology who dont understand it always fall victim to shortsightedness. if it works, ship it, and damn the consequences. well now we're paying for it in full. forget myspace, the whole www is a cesspool of cross site scripting, sql injections, activex exploits, and a whole plethora of goodness. at this time, i would like to take a step back and say unix has a shitload of exploits too. but still, my point remains valid.

now you have armies of web developers feeding the flame and i am afraid i will soon become one of them.

side effect as goal

you know, its funny. i find myself thinking and saying this a lot. i wonder when in the course of history did writing software become a way to make money. people started out writing good software for other reasons, whether to scratch an itch or to impress the ladies, they didnt start writing it to make money. now they discovered that along the way, they ended up making some serious fuck-you money, which is great of course. now all these gold digging son-of-bitch posers jump on the bandwagon to try to create the same side effect, but fail miserably and they wonder why. i dont blame them one bit. you see people getting rich doing something that you think you can do, you're going to fuck over anyone to get there first.

what they fail to realize is that to make money on software, it has to be good software, and you dont go about creating good software because you want to make money. i swear to christ that these people have on their requirements doc "make money" as the number one item. in order to make money, you have to not want to make any in the first place. that's funny. unfortunately, these people do stay alive and do well in the economy that has been created from good software. they tag along inside big companies or just get money from other people who think they will eventually get some fuck-you money from their efforts. few of these things rarely work out.

and yes. there are piles of books on the subject of how to write good software and all that junk. its not going to matter to most of the people i speak of. they're so lost in the forest that they just keep hacking away (pun definitely intended) and miss the big picture. no amount of schooling is going to set these people straight.

so i guess my advice to people who want to make money. DONT TRY. just try to do something good. the money will come as a side effect, or it wont. either way, it is your best shot.

a new appreciation for managers

we've all said countless times how useless our managers are, or how our productivity would increase 10-fold if we didnt have to report back to those pesky overseers. i still think that's all pretty true, but the reason isnt because managers are unnecessary, but rather, managers arent doing their jobs well. i dont really blame them, because its really really hard.

one of the major misconceptions that hackers turned software engineers have is that when they're at work, they're building a product. somehow, people get a notion in their head that when they're at work, they get to do things that are "fun". since everyone expects this, the industry has in turned responded with telling its engineers that they are suppose to be having fun. this is just not true.

the first distinction to draw is that when at making a product, you start with a requirements document that you implement and go from there. if you are just hacking on something yourself or even on an open source project, you can do any number of "cool" things that you want. in a corporate environment, you have a deadline, a feature set, and the quality of your product. managers fail to juggle their resources sufficiently to meet all of those things. an effective manager has to know what he has and make it work for the product. they have to make tough decisions about the number of features to be included, while measuring intangibles like code quality. most importantly, technical managers need to coordinate his team, and that means having people skills. they need to read people, recognize the type of people they have, and make them work effectively together. there is often an inverse proportion between the amount of social skill someone has, and their technical ability. while many of you may be outraged at this point, i maintain that its true.

with all the managers floating around, there are bound to be some pretty crap ones. that's when deadlines slip, people get overworked and assigned to tasks they are not suited for. crucial features may be left out and corners may be cut. in the end, you end up with some piece of shit software that the developer then say, mmm, when i wrote something similar to this myself, it was so much better. well, there are reasons for that. this goes into the whole cathedral and the bazaar shit and software engineering theory that i wont go into and assume you are familiar with.

my point is just that, i recognize now, more than ever that managing a software project is hard. if its not some architecture decision, then its some high-school-like personality dispute. its a wonder how any good software gets shipped.

hopefully, this will last

in the past, i have tried to maintain a blog, but to no avail. i'm not sure why that is, but it could be because i had no focus on a topic. i simply wanted to blog about my "life". that obviously was not enough to keep me writing. the topic of this blog i'm hoping will be my experiences with "why writing software is hard?" this doesnt just include my experiences with writing software, but also my discoveries with other people's software.