Dec 21, 2014

Exodus: Gods and Kings

THIS movie should really be called Exodus: Moses’ Struggle with God.

Early on in the movie, Moses (at the time an Egyptian general), travels to the city of Pithom to investigate complaints about the Hebrew slaves from the Viceroy assigned to the city. They have a conversation in which the Viceroy mocks the Israelites, saying the very name itself means ‘one who fights with God’. Moses corrects him and says it means ‘one who wrestles with God’. Personally I would use the word ‘struggles’ instead of ‘wrestles’, but the point is that that exchange foreshadows Moses’ relationship with God.

As much as the movie is about the suffering and deliverance of the Hebrew people in Egypt, and about how Moses finally finds some measure of happiness in exile with his wife and son, it is more about Moses’ relationship with God–a very personal relationship, almost an equal partnership at times.

Moses is very clearly an unbeliever–he has grown up surrounded by the Egyptian religion with its pantheon of gods (not to mention Pharaohs), and has remained unconverted by any of them. He finally decides to follow God (their relationship, as I mentioned, doesn’t even look like any kind of deity worship we have today) because that’s the only way he sees to save his people.

Moses sets out to save his people from Pharaoh, but his attempts don’t have much impact, while Rameses’ retaliation seems expressly designed to dispirit and demoralise, frighten and terrorise, the Hebrew slaves. Like any clever slaveowner, Rameses avoids doing much real damage to his property, while still punishing them enough to (in his eyes) frighten them into submission.

That it doesn’t work is evident whenever we see the faces of the Hebrews as they observe the injustice. That they persevere in the face of it all seems like the real miracle to me, not the plagues and the cataclysms. Early on in the movie, Moses tells the Viceroy that you can tell a lot about a man by looking into his eyes. When you look at the faces of the Hebrews, it seems as if God is behind their eyes looking out at you.

God in the movie is a wrathful God. A God of vengeance. There are no two ways about it. He cannot coerce men nor preempt their minds, having given them free will. But He can and does preempt the natural order and the natures of the beasts and insects, and causes them to rain down upon Egypt in plague after plague. The punishment is intense, the suffering severe. Moses grows frustrated with it. ‘Who are we punishing?’ he asks. When God acts upon the face of the Earth, his action is like a giant hammer that smashes down upon all, without discrimination.

The final plague is what finally threatens to break Moses’ resolve: ‘No! I cannot be a part of this!’ In a final act of wrath, God reveals that He has heard Rameses’ threat to kill every Hebrew infant, and He will take the life of every first-born child of Egypt, unless they are in a house whose door is marked with the sacrificial blood of a lamb. Of this escape only the Hebrews are warned. Finally, God plans a way to discriminate between His people and their oppressors.

Knowing that there will be a massacre of innocent children is a hard thing to take. Yet the God of the Old Testament has many times been wrathful. He has taken perhaps millions of lives as punishment for evil. The Passover is the first time that He has actually planned out such a massacre with a human general and has chosen who will live and who will die in such a targeted way (well, perhaps since Noah, but then the Ark was also a much cruder means of selecting survivors than what they did for the Passover).

The way that God reveals Himself to Moses is interesting. Before I watched the movie, I heard somewhere that God took the form of a British public schoolboy. This intrigued me because it meant that Ridley Scott subscribed to the idea that God, being eternal, experienced all moments of time (past, present, and future) simultaneously and could thus introduce anachronisms by appearing in one time period as something from a completely different time period.

But ... I probably took that too literally. It didn’t actually happen. Instead I saw something just as interesting, but in another way. God asks Moses who he is, and Moses replies ‘A shepherd’. God says, ‘I thought you were a general? I need someone who can fight.’ As if He is a wartime leader recruiting a general to lead an army on a front. Which of course He is, and which of course is what He needs Moses to do. The portrayal is very, very interesting when you think about how Yahweh, the Hebrew God, started out in the oldest stories as a legendary warrior hero and leader of his people. In some stories, He had once been in the same position that He now wanted to recruit Moses to.

Many times throughout the film, Moses struggles with God, perhaps even chastises Him as being too cruel and vengeful. After four hundred years of watching His people suffer, apparently God has some pent-up wrath. Ultimately though they agree on one thing: the people of Israel need protection and guidance to find their way home.

Jul 7, 2014

Easily Authenticate when Pushing to a Git Remote

SOMETIMES when you’re working with git repositories, the remote doesn’t support pushing and pulling over SSH. It only supports HTTPS. Or you don’t have an SSH key properly set up, or you do but it’s not working for some reason. Whatever the cause, the upshot is that whenever you push, you’re forced to type in your username and password. This sucks, understandably. But do you use a password manager, like KeePass, on Windows by any chance? Because if you do, you can authenticate absolutely painlessly. Here’s how.

Set up a KeePass entry for the git server you’re pushing to. Let’s use GitHub as an example. Create a GitHub entry with your username and password, then make sure the entry has these two properties:

Auto-Type: {USERNAME}{ENTER}{PASSWORD}{ENTER}

Auto-Type-Window: Git Bash

If you’re using git on Windows, most likely you’re using Git Bash. Of course, if you’re using Posh Git, then just change the window name to whatever is appropriate.

Now, when you do a ‘git push’ and the git remote asks you to authenticate, simply press your global autotype combo (by default it’s Ctrl-Alt-K) and KeePass will log you in immediately. No SSH necessary.

That’s convenience.

May 13, 2014

Stack Overflow and its Discontents

LIKE many others, I’ve come to rely on Stack Overflow (SO) as an amazing repository of technical knowledge. How did it become so comprehensive? My guess is it was the ‘many eyes make bugs shallow’ principle. I.e., many contributors building something together, a lot like Wikipedia. SO is nearly always great when you’re searching for an answer, but not that great when you’re trying to contribute and build a profile for yourself among the other members. I’ll explain why but first let me clarify something as I see it: SO may look like a simple Q&A site, but it’s really a permanent knowledge base with the knowledge organised in the form of questions and answers. That’s important: the Q&A format is just that, a knowledge delivery mechanism. So with that in mind, we can examine how that affects peoples’ actions on the site.

Lots of people have discussed why SO punishes or rewards contributors the way it does, but one widely-held belief is that there is a subset of users (usually the moderators and other high-reputation users) that is intent on seeing SO’s core mission carried out: that the site becomes a repository of useful and generally applicable questions and answers. To keep it that way, this subset performs triage on the questions that are asked: they apply judgment calls on whether or not the questions are good ones. When you’re a beginner, there are no bad questions. But when you’re building a long-lasting repository, there are bad questions.

Generally speaking, bad questions on SO could be any of:

  • Duplicate of an already-answered question
  • Not phrased as a question
  • Not clear what the asker wants to find out
  • Asker shows no signs of having done any research or made any attempt to solve the problem
  • Question is about a specific corner case which can be easily solved if asker understood a more general principle
  • Code examples are incomplete and can’t be compiled, error messages are not quoted verbatim but only vaguely described
  • And the other end of the spectrum: many screenfuls of code or log output pasted verbatim, in entirety, without any attempt to zoom in on the source of the issue.

Any of these questions will annoy the mods because they’ll invariably get answered (because people are incentivised to answer no matter how bad the question), and then those bad questions and their answers will raise the noise level and make it difficult for people trying to find answers to the genuinely good questions. (Good search features, and even Google, can only take you so far.)

So with this in mind, we can start to understand the mods’ behaviour of seemingly penalising usually newer users of the site, those who haven’t absorbed the culture yet and are treating SO as a source of answers to one-off questions. It’s not–the questions are meant to become part of a knowledge base on the subject and potentially live forever. Under these conditions, it’s very difficult to justify questions with the above bad qualities, especially if we can guide the new users towards improving their question quality (and lots of people are trying to do this).

So, remember, the mods’ goal is to build a generally-useful knowledge base. With this as a given, the questions (subjects of knowledge) that are of a low quality will tend to get weeded out: either by being downvoted, or closed. The people who’re doing the downvoting and closing don’t have the end goal of punishing the askers; their goal is to weed out the bad questions. That the askers lose rep points is a side effect of the voting and rep system. Which is fair: if my peers judge me as not contributing good material, then I should have less cred. But the primary goal on everyone’s mind is having good material on the site, not punishing me.

Having said all that, I want to address the fact that votes on each question and answer are essentially on an infinite scale going in both directions. So, a given question or answer can potentially be upvoted or downvoted many times over, and every one of those votes affects the poster’s rep. But the effects on rep are all coming from a single posting. That’s skewed, because users on the site are more likely to see higher-voted questions and answers than they are to see lower-voted ones. That’s simply how the search facilities work by default: understandably and helpfully, users get to see highly-regarded content before they see the dregs of the site. But this means that highly-upvoted content will always get more exposure, and therefore continuously be exposed to more upvotes, while downvoted content will get less exposure and less downvotes. This skewness is disproportionately rewarding the experts and inflating their rep.

Let’s ignore the downvoted content for a second and think about the upvoted content: it is continuously getting upvoted, just for being there. Meanwhile, the person who posted that content could very well have not made a single contribution after that (taking this to the logical extreme). That’s an unfair advantage, and thus a bad indicator of that person’s cred in the community.

It’s clear at this point that the SO rep system is not going to be recalibrated yet again (barring some really drastic decision) to fix this bias, so let’s imagine for a second what a rep system would look like that actually did fix it. My idea is that such a rep system would reward (or punish) a user for a single contribution by a single point only, to be determined as the net number of up (or down) votes. So, if a post had +1 and -1, the reward to the contributor is nothing. If the post has +100 and -12, the reward to the contributor is +1. And if the post has +3 and -5, the reward is -1. If there’s a tie, the next person who comes along has the power to break that tie and thus ‘reward’ or ‘punish’ the contributor. Usually, of course, the situation won’t be a tie–usually there’s pretty good consensus about whether a contribution is good or not (to verify this, simply log in to Stack Overflow and click on the score of any question or answer on the question’s page–it’ll show you the upvotes and downvotes separately).

The sum of the net effect on reputation from each of a contributor’s posts shouldn’t become the final measure of their profile rep, though. That doesn’t give the community an easy way to tell apart a person with +100/-99 rep (a polarising figure) from someone with +1/0 (a beginner, an almost-unknown). Instead, users should be able to judge contributions as +1 (helpful), -1 (not helpful), or 0 (undecided). And each net reputation change from a contribution should form a part of a triune measure of rep: percentage helpful, percentage not helpful, and percentage undecided.

The three parts of the measure are equally important here. The vote on the merit of a contribution is nothing but a survey; and any statistician will tell you that in a survey question you must offer the respondent the choice of giving a ‘Don’t know’ answer. In this case that’s the ‘undecided’ option. If we don’t offer this option, we are missing critical information about the response to the contribution–we can’t tell apart people who simply didn’t vote, or those who tried to express their uncertainty in the question/answer by not voting.

This way, everyone immediately sees how often a particular user contributes valuable content, as opposed to unhelpful or dubious content. And the primary measure of rep is therefore not something that can grow in an unbounded way: the most anyone can ever achieve is 100% helpfulness. That too, I think, should be quite rare. The best contributors will naturally tend to have higher helpfulness percentages, but it won’t so much a rat race but rather a level marker within the community, tempered by their levels of ‘unhelpfulness’ or people’s indecision about their contributions.

So much for user profile rep. I do think that the scores on questions and answers should behave in the traditional SO way: all votes should be counted as individual votes, instead of (as with user profile rep) being summed into a net positive/negative/undecided vote. The reason for the difference is that the votes are the measure of each contribution’s merit; and if (for example) you have two similar contributions, their vote scores should be a good way to judge between them. Again, vote score should be presented in the form of a three-fold percentage measure of helpfulness, unhelpfulness, and undecidedness (with vote counts available on, say, mouseover). This keeps conformity with user profile rep and puts an upper bound on the score shown on each question or answer. The reason why this is a good thing is that most of the time, to a site user, unbounded scores are simply extra information they won’t really process. The site itself can easily search and present content in order of most highly-voted; but the reader just needs to judge merit on a fixed scale. Anything extra is just cognitive burden on the reader.

So to recap, if we’re to implement an unbiased user profile rep, we need to count the net helpfulness of each contribution once only. But for content scoring, we can count each vote individually. And to present everything in a uniform way and with low cognitive burden, we should show all scores as a three-fold measure of percentage helpfulness, unhelpfulness, and undecidedness.

Once we have this system in place, we can start doing interesting things, like automatically closing questions which sink below a threshold of, say, 10% helpfulness (they’ll start out with 100% helpfulness because the original asker’s vote is automatically counted as helpful–otherwise they wouldn’t have asked). And we can do away with reputation loss from downvoting, since a downvote will usually have no effect on the recipient’s profile rep, and only one unit of negative effect on the contribution being downvoted.

Achieving an unbiased measure of rep is tricky. But I think we can do better than SO’s current system by rebalancing the ‘rewards’ and ‘punishments’, and bounding the primary measures between 0 and 100% helpfulness so that we don’t end up with another Jon Skeet on our hands (I kid–Jon Skeet is great :-)