Hacking with the Tao
Zen Code (aka "Hacking with the Ta0") v.20151208 by \0xDynamite see the original version at http://wiki.hackerspaces.org/Hacking_with_the_Tao or http://wiki.hackerspaces.org/User:average/Hacking_with_the_Tao, otherwise this document has moved to the Singularity project.
NOTE: This has yet to be re-formatted.... Please help.
STUB Need to integrate this knowledge:
- A circle with four points: architecture, management, engineering, construction
- These four ideals: modularity, readability, efficiency, correctness,
- Four quadrants of use: PositionOS:epistemology, metaphysics, data-visual processing, math.
- on two axiis: re-useabiltiy (readabie and modular); elegance (correct and efficient)
- one-meta: Zen.
Programmers and coding generally fall into one of these four rough categories:
- participating in the programmer ecosystem (Free code, ObjectOrientedProgramming),
- performance-based for scientific computing,
- secure databases for business use, and
- fun and learning (toy problems, games, and such).
The purpose of this guide is to be so abstract as to force you, the experienced programmer, to think of programming language design itself as a part of your path -- that the programming language is only a tool for the process (like a awl or chisel) and the programmer is a type of craftsperson. The reason is because the data and their relationships are growing faster than any programmer`s code. No language has figured it all out yet. Are you up to the challenge? There are exabytes of data out there now, mostly feral. As such, this is for those who want to go all the way and forge it into something unbeatably awesome. For this, OOP, the GlassBeadGame, and this Guide was made.
Like any noble path, you'll find many counterfeits who claim to be a master. They may code like a pro and speak as if they have the Way, but when the glamour fades, you'll be left with cute snippets, akin to wearing a pocket protector when you need to build a 5-story building or giant sheets of steel when you want a Ferrari. Because the truth is: most people don't know how to manage the complexity of data.
The way to mastery is a long, winding path. Computer programming has moved away from computation and more towards abstract symbol processing. Being skilled at math won't make you a master. Also, don't be fooled by the glamour of Jabbascript. Those guys are like chickens in a cage filled with many levers to pull where thousands of pretty, colored pellets drop out -- but there's always some wizard working behind the curtains.
In any event, there are few real masters. True masters have tuned themselves to the machine, not the other way around. They also seem to always have a noble purpose. You don't have to follow me either, and should test my words with your own experience as the art evolves. Just bring this little scribble with you along the way, and then you'll see for yourself...
The computer is your unwavering teacher and will hone your critical-thinking skills like no person can, as it allows no sloppiness, gives no favor, and has unending patience. If you're content with mediocrity, get your certification from Microsoft and slap it on your resume or follow the Jabbascript crowd and be done. Most people succumb to sucking it for the anti-Christ, generating gigabyte data silos that no one else can use, or they content themselves with an academic understanding and spend their life writing theoretical papers, never having done the work of a real programmer. Those are hardly noble enough efforts worthy of the most powerful product industrialism has ever produced. If towering monoliths of data are whetting your appetite, click your browser`s "back" button now.
Otherwise, plan on growing your neck-beard (or female equivalent) and let's go.
You shouldn't go into battle without knowing your opponents. On the way to mastering the machine, you'll understand there are only two real adversaries, always alongside you. Like Death, though, they are always invisible and ready to consume you.
ADVERSARY #1: RushingToTheFinish. You'll be inclined to get a program to work as soon as possible. Everybody likes to see immediate results, but down the road, this short-term savings won't be extendible--if it's even comprehensible. Most code should get re-used or appreciated, otherwise why write it? We're in the Internet Age here. I'll appreciate your slim code for your ultra-elite application later -- after you re-write it modularly so I can use it for my applications. Meditate on this instead, grasshopper. Hidden blow: Prematurely optimizing your code.
The best counter-weapon for this battle is TestDrivenDevelopment.
ADVERSARY #2: CruftMonster. You wrote some code awhile back. You're not sure how it works anymore, so you don't want to change it, right? You pussy-foot around it, as if treading on thin ice of the deep lake in which this monster resides, worried it will crack at any moment. That's called cruft. I'm assuming brevity is a shared value in this guide. You're a programmer, not a shill working for LOC, right? So, bite the bullet. Guided meditation here. Hidden gotcha here? Believing in all your working code. Nothing spells "cruft" like True Believers.
The best weapon for this battle is RefactorMercilessly which you can draw out of your bag of ExtremeProgramming.
BTW, the false prophets mentioned at the start are like something incomplete. They are as demons enslaved by these enemies who now feed off of new meat -- best keep your distance. This short little guide is aimed to be everything you need.
It is always best not to enter an arena unprepared. Without exaggeration, these two have captured many lost souls. Those who succumb to the first, hardly make it out of Scripty -- that mythical land where computers run like magic. They never even know they've been victimized. Those who succumb to the second -- help me Thor -- rarely, if ever, come back. They curl over and merge into their cubicle, like pupae. Got it?
After many decades of wrestling with these enemies in the field, some powerful allies have emerged. Keep them close to you at all times. They should work on you even when your work seems done. Despite their name, you shouldn't assume they're friendly. They are primal forces for forging YOU, the Apprentice. The value of the allies exists (and can be used) independently of my linked commentary. Treat them well and they'll stay lustrous. Used properly, the allies harness the opposing forces and hone you into a Master....
Being somewhat independent of your personal thoughts, the allies are "crucibles" -- they exist in a complex contradiction that YOU as the programmer must resolve. They are koans. There is no set of rules or guidebook in this terrain because you'll be forging the path, otherwise you'd earn nothing. Meditate on them.
- Crucible #1: NameFu :: ClotheYourData vs. LeaveItNaked.
- Crucible #2: LogicNinja :: SeparabilityOfDomains vs. ZeroSuperfluousNames.
- Crucible #3: ModularityDojo :: TightlyCoupleInternally vs. LooselyCoupledExternally //LooselyCoupled (internally) MinimizeDependencies
- Crucible #4: TRON :: ReduceMemoryFootprint vs. MinimizeWork.
What makes good code: Readability, Correctness, Epistemological Order, Efficiency. These are related, respectfully to the four allies above.
The complete toolkit for forging your program are Divide, Encapsulate, Conquer, and Optimize. The word "conquer" should be considered in the sense of "ratiocination": the application of exact logic. You can cross each of these with each other and gain useful insights. So, for example, Optimize x Conquer and you have "profiling your code" (see programming cross-products). Apart from your programming environment (which should be a reflection of your relationship with these four tools), there is nothing more.
Those who put faith in abstractions have confused the path. So let's get this straight at the start: you're on a vonNeumann machine not a Symbolics. That means computer programming consists of loading a simple programming language statement and any data that goes along with it, executing said statement, and continuing forward excepting on some condition or error otherwise, and repeating this sequence. It's like a little Turing Machine, the basis for much of Computer Science, that. It may or may not finish. All that's to say: Let's not go on a joy ride upon the language-du-jour or get too academic, 'k? We will, at some point, be worried about memory and CPU performance as all good programmers should, we're just saving it to last -- just as Master Bentley taught us.
Divide and Conquer you know by now and are the basics of every computer science course. Encapsulate was always implicit, however, in your programmer's reference. It is the honing of useful routines into re-useable components. Routines and data structure that have proven their usefulness generally get their own syntactical token(s). Python, for example, uses squ[a]re brackets for what used to be complex linked-list data structure. It is ultimately the evolution of the programming enterprise in the age of collaboration. Each function or routine provided for you (or class and object) represents a solution and higher-order synthesis from those who came before you. It has rather peaked with the ObjectOrientedProgramming paradigm, but the covert purpose of this manual is to get beyond that plateau. Optimize was also always there, but it went as a generally-unspoken value among the masters. With high-level languages and web programming, optimization seems to have gone extinct, but it remains the true sign of a genuine master. In isolation, optimization can't meet the goals of OOP. So these four tools should be seen as mutual-aids.
So, then, the full stack for coding consists of various layers from the abstract to the concrete. From the top, you have the User/Programmer --> Interpreter --> Compiler --> Hardware. The word "compiler" has been a bit ambiguous, much like "interpreter", which in this context is not to be equated with "interpreted languages", but more like "parser".
For the four terms listed previously, each encloses a semantic network of sorts, going from higher-order to lower-order something like this (respectively): concepts --> expressions --> imperative statements --> digital logic. The Programmer is composed of concepts, the Interpreter of expressions, the Compiler of imperative statements, and the Hardware of digital logic. Each pair has a force which binds them.
The force that links or binds User to Interpreter is called programming, from Interpreter to Compiler is the lexer, and from the Compiler to the Hardware you have the loader at program execution which is generally determined by your Operating System or environment. Beyond that, should be the unwavering execution of physical law.
An assembler is a compiler in this definition, taking imperative instructions (not expressions note) like "MOV AX, 1" and putting it into machine code (like 0x0A01). This binary machine code operates through the logic gates on the CPU in whatever complex ways the manufacturer has stamped upon it and specified. That whole endeavor is an art and science in itself, called "computer engineering". Additionally, an interpreted language like Apple BASIC has to be compiled at the point you type "RUN". This is true regardless of whether the language is called an interpreted language because the CPU can't process nor ever sees ASCII characters (like "GOTO 100"), it only deals with 1s and 0s (or *ultimately* voltage gradients).
So that's the stack. Now what do you do?
Well, the Grand Process of mastering the machine has already started in several ways. 1.Your language designer already divided the problem of getting computers to do your bidding and you have learned how to use that power. 2.Secondly, the history of computing has already conquered the whole domain by separating I/O from Processing in order to make a General Purpose Computer. This is a giant help and saves you buttloads of time wiring I/O junctions to your panels. 3.The document you have here is your path to optimize the process of utilizing the machine to it`s full potential. 4.Lastly, you are the synthesis who's going to make it happen!
Practically, there can be several different "breakdowns" of how you go about structuring your program, but if you're constraining yourself to a particular programming language, chose the axis in which your language was designed: concurrent vs. sequential, file-based vs. interactive (i.e. interpreted), procedural vs. imperative dimensions, et cetera. If you're not constraining yourself, choose Python/C and make it easy on yourself.
For Object Oriented Programmers, you should check out the the Gospel, so you can be sure of salvation. Though you may not need me holding your hand though the process, I'm going to try to describe it through a new lens.
In essence, your language should constrain you at the bottom by forcing syntactically-valid expressions while YOU constrain your program at the top, keeping the purpose for your program in mind. That leaves the middle with the most degrees of freedom. So the first thing to do is to start there, writing boilerplate and your first architectural drafts. Write some tests which you need to accomplish your task and then the minimal program that will compile and run. It's okay if it fails, because you've gained some knowledge. Continue fixing your program until it passes all your tests and does what you want. The general idea is that you're going to keep constraining yourself by dividing the problem with your architecture, while the language keeps constraining you at the bottom forcing you to conquer your logic. In the middle, you encapsulate -- forming categorical groupings (functions, objects, modules) -- and hone your code towards greater precision until it works and becomes a work of art that other programmers can appreciate and re-use. That's the whole process in a paragraph. Click on each link for the tour.
You may have heard of TestDrivenDevelopment, but the real win are tools like Python's DocTests. The goal is re-useable code -- that means trustable code and readable examples are other programmers` best path to understand what your code is supposed to do. Doctests give you that and you get code-testing as a bonus -- truly great.
Note that I haven't said anything about revision management, because it may or may not be how the allies work with you. Remember they are what's going to hone you into a master -- not the language and not your tools. Revision Management helps you with SeparabilityOfDomains and TightCoupling, but those are only half of two different crucibles. They can also help you with CRUCIBLE 4. You have to figure out whether that's your path or not.
As for software versioning, use major numbers for incompatible changes only and the minor numbers on the right of the decimal for minor, compatible changes. Stick with this and there's less explaining to do. You can have any number of digits to the right of the decimal point, so you won't run out of minor versions.
When you apply the concept of using and understanding the allies while you code, you'll see that the four allies are directly correlated, respectively, to these four processes and have always been working on the programmer's soul. Any stagnation you have is because you've let one of the enemies into your practice. If you find yourself at a programming or personal impasse, explore the links under "adversaries". But, when you're looking for an environment for other like-minded coders, to master your techniques and work on some awesomeness, look into hackerspaces. Or, perhaps drop it all for awhile and try your skills at Hacking the Law. Ultimately, programming is just a bridge, a means to an end where computers may not even be necessary anymore. It's at that point (and that point only) that you have to decide whether you've reached the end of your path.
Lather, rinse, repeat...
You have working code. It's documented; includes tests so others know it will work as they expect. It has reusable functions, objects, and/or modules, demarcated from internal functions and objects by some syntactical flourish, and it does something interesting or useful. So...
The right outcome of the battle between You and RushingToTheFinish is Maintainability (of your code) and personal Reserve. The right outcome of the battle between You and the CruftMonster is Parsimony (read: simplicity + harmony) and personal Mastery.
If you make it to the end of this long road, you'll have found that the enemies and techniques of programming are applicable to life itself: don't rush and beware of your own cruft. You will have achieved Elegance and have become the Victor!
Now sacrifice your Amulet of Yendor and let's go!
Special thanks Steve Summit who wrote the C FAQ, to the denizens of RefactorMercilessly, TimPeters for imbibing Python with some powerful Zen via DocTests, Bjarne Stroustrup for my concept of encapsulation, Niklaus Wirth for the concept of modular programming, and many other old-school programmers who lit the Way. And, of course the Tao te Ching. WikiWords and eXtreme Programming are documented at the wikiwikiweb: http://c2.com/cgi/wiki. See the PositionOS project here at GitHub for more directions on how to proceed towards epic vistas!