[community manager]Currently at Eurecat Data Science we are developing a research project on evidence-driven community management. Our long-term aim is to provide better tools that allow community managers to attain their goals and to better react to social media communities. Our short-term aim is to understand the goals, methods, and tools used by community managers.

We are looking for people who have worked in the past 3 years building or developing an online community around a brand, product, cause, or person. We are conducting brief phone interviews (30-45 minutes), with open questions. Interviews are confidential and done in the framework of an informed consent document offered by us to the community manager before the interview.

Our intention is to give interviewees preferent and/or early access to our research results. If you are interested, please sign up. If you know someone who might be interested, please share with him/her this invitation. Thank you.

On April 25, 2016, the Cryptome foundation disclosed that a large zip archive containing what seemed to be private data from hundreds of thousands of customers of the Qatar National Bank (QNB, the largest bank in the Arab Gulf in terms of assets) had been posted online.

The archive

The archive (510MB compressed, 1.4GB uncompressed) contained about 15K files in many directories. The most obviously significant were (1) a set of database tables, and (2) a directory named "Folders."

The directory named "Folders" is a set of small dossiers of about 100 "notable" people in Qatar. It includes several sub-folders named "Al Jazeera" (the name of the Qatar-based news network), "Al Thani" (the name of the ruling family), "Police, security" and so on. Inside each folder, there is one or more files containing a mixture of data extracted from the QNB database (such as account number, passport number, and so on), and information from other sources, that varies from links to online profiles to a photo of the person, usernames and passwords. I say "notable" in quotes because the classification of people into folders here is a little bit dubious, for instance with some people named as spies when they are unlikely to be so.

The database tables contain profile information of about 300K-400K customers. This includes name, nationality, passport number, national ID number in Qatar, e-mail address, and physical address. It also includes card numbers, expiration dates, and account numbers for about 800K-900K debit and credit cards. There is some duplicate information between a set of main files and a set of back-ups, so the number varies depending on which files you look at.

My involvement

I worked in Qatar from mid-2012 to mid-2015 as a scientist in a national research institute. From my time in Qatar I knew that QNB is the first choice for many foreign workers, so the data leak had probably affected many of my colleagues who still work in Qatar.

After downloading the archive I did a quick search for e-mail addresses in my former institute, I found 4 people I knew, including their full names, which was a strong signal that the archive was authentic. Additionally, there were things that you only see in Qatar (such as writing "QATAR FOUNDASHON" instead of "QATAR FOUNDATION"). I alerted my colleagues, but didn't want to look more into the archive; I was not interested in learning anything private from them, and given I did not find my own e-mail address, I assumed this was old data from before I went to Qatar.

However, hours later and after being alerted by a friend, I did find my account in the archive. My e-mail ID was not included in the main files, but in one of the backups, and my national ID number in Qatar was in multiple places.

I looked more carefully in the archive and found the names of people I knew, but not their e-mail addresses. Then, I decided to create a tool to let people search for themselves if they were present in the archive or not.

The "QNB Verifier"

The tool I set up on April 26th was very simple. First, I hashed all the e-mails and Qatar national ID numbers I could found and uploaded the hashed identifiers to my web server. Second, I wrote a small Python script (my first program in Python) to receive a hashed user input and compare it against the stored hashes. Third, I used an Javascript implementation of MD5 to hash the input in the browser of the user.

I set-up a page to give access to this tool to users, and warned them that their hashed input could be observed while in transit, but indicated that no plain-text personal data would leave their browsers. I also made it clear I was not hosting the leaked files. I kept no logs of the user input but my service providers has statistics of how many people accessed the endpoint of the service. About 10,000 did so during the 5 days or so that the service was active.

In the meantime, QNB continued calling this a "social media speculation," a line they maintained for several days.

The denial-of-service attack

From April 27th onwards I started to get thousands of requests per second from a few IP addresses. This increased substantially the CPU usage of my account. My hosting provider froze all my websites (i.e. started serving static versions of them) except for this verification service (!) and warned me that if this high CPU usage continued they will suspend my account.

I added the IP addresses of the attacker to a blacklist but s/he kept on changing them. I spent hours playing whack-a-mole but fortunately for me the attack was not very sophisticated, and eventually my attacker gave up and stopped changing IP addresses. That brought my traffic down to normality and my websites were unfrozen.

The legal threats

On April 30th the service was featured in Doha News. Doha News is an independent online news service that operates from Doha. They knew me from my time in Qatar and had covered some of the work on predictive news analytics that my team did for Al Jazeera. They linked to the verification service and interviewed me over Twitter to know about my reasons and how this worked. The International Business Times also followed up on this story.

On the same day, I received an e-mail from Hispasec, a Spanish IT security firm. In their e-mail, they wrote in Spanish:

I am .... from Hispasec Systems, an international IT security company, in legal representation of QNB Bank from Qatar.

The entity had reported your 'qnb-ver' service for us to proceed to shut it down, because they understand it can be being utilized to collect e-mail addresses or bank account numbers:

URL: http://chato.cl/2016/qnb-ver/

We understand this is a special situation, but due to its importance and the requirements of our customer, we must emphasize that you must disable this content from your website.

Many thanks for your collaboration.

I responded a couple of days later and had some exchanges with them. I basically stated that I was not hosting the leaked files and that I was not collecting any data. They reinstated their request and warned me about the importance of promptly removing my service to "avoid to face any legal action concerning the spreading of the public damage and defamation." I have to say I never felt this was something to be concerned about; in Spain you are fairly safe unless you speak against the king.

Censorship

The Internet has always been heavily censored in Qatar, and many different content categories are not accessible from the country. Censorship is, however, quite easy to circumvent with VPNs, and many people use them.

On 2014 or 2015 they set-up a censorship page, that you can see in censor.qa that shows a cartoon and explains that you have accessed a page that contains prohibited materials. This makes censorship more "friendly," I guess.

My page was added to Qatar's blacklist on May 1st, 2016. I set-up an alternative, https-based access on the same day and publicized it, people continued being able to access the https-based service.

Incidentally, on May 1st, 2016, Qatar National Bank issued a statement indicating that the leak had only affected a "a portion of Qatar-based QNB customers." I would say that "portion" is very high, close to 100%. I did not hear about a single QNB customer that used my service and did not find his/her national ID or e-mail in the leaked files.

Downfall

On May 2nd QNB contacted my hosting provider to report a "phishing" attempt from my account. Phishing is when scammers send people e-mails asking them to follow a link that looks like a bank's homepage (or other institution), so that the victims enter their credentials in the fake site, and the scammers can steal them.

In response to the phishing complaint, and without any examination, my hosting provider proceeded to close all the websites hosted in my account, including my personal site, my research site, the website of my upcoming book, the personal website of my wife, an environmental portal that she maintains, and others.

I replied to the report with a detailed explanation that no personal data was being hosted, and no data was being stored, moreover that the service was designed to make this impossible by hashing the personal data on the users' browser. I received an automated response that pointed out the offending files were still there. Basically, a human would not look at the ban until the files were removed.

So, I had to shut down the service, and it took me about one day to get my hosting provider to bring the websites back into operation. I did not want to fight this. Their business is to have customers that don't require any attention, and if you use too much time from the support desk, they can easily revoke your contract and send you with a nice backup in search for another hosting. In my past experience that is 2-3 days of work that I don't have.

What did I learn?

In my opinion, data leaks should be regulated similarly to work place accidents. Companies should have to file an official report indicating what happened and who was affected. Most importantly, customers must be told the truth, but they get nothing, or lies, or partial truths that are not helpful. This is not a problem only in Qatar (many others have done exactly the same), but is definitively made worse by Qatar censorship and heavy-handedness against any "troublemakers."

I learned a couple of things. First, the public is very weak in these situations. You cannot know if your data has been leaked or not, Downloading the archives and verifying yourself takes some technical knowledge that, while superficial, is beyond reach for many people. The are services that can help you, such as Have I Been Pwned?, but they use only e-mail IDs.

There are way too many choke points, from legal systems, to censorship systems, to hosting providers, that can be used to silence anyone. Also, if I had been in Qatar I would have risked jail time followed by deportation, and most likely I wouldn't have done this under those circumstances.

Second, we are also quite strong, for many reasons. The leaked archive is out there and is not going to disappear: you don't get the genie back in the bottle on the internet. That is good because people can still verify if their information was stolen, but bad because the archive contains personal data and hence the potential for abuse is huge.

We are also powerful because we are many and we can move fast, way faster than what many corporations and governments can.

What else did I learn? If I had to do the same thing again, most likely I would find a secondary hosting provider and do it there; the price of getting your main hosting account attacked is too high, and while I was expecting QNB would go to my hosting provider, I thought they would at least look at the complaint to examine its merits. There are many hosting providers that are better suited for this than the standard ones. I would also set-up this under https from the beginning to make it more resilient and secure.

Everything else, I would do the same. It was the right thing to do, and I'm glad I was able to help.

From New Scientist, April 4, 2016

... In Zambia, there are roughly 27,000 new HIV infections a year, according to UNICEF, and 40 per cent of these are in those aged 15 to 24. With people constantly texting U-report for all kinds of HIV information and advice, the automated version uses machine learning algorithms to sort messages into eight categories: symptoms, HIV testing, treatment, pregnancy, transmission, prevention, definition, and male circumcision.

To train the system, Patrick Meier, then at the Qatar Computing Research Institute in Doha, and colleagues fed in at least 50 messages for each category that had been selected by hand, and asked it to identify patterns that it could then use to do the sorting itself. As well as how to handle typos, the system learned to cope with textspeak such as “HOW 2 AVOID SPREADING HIV/AIDS 2 OTHERS?” and “I feelin bad becoz im th only one wh hs hiv wht shld i do?” ... More »

Muhammad Imran, Patrick Meier, Carlos Castillo, Andre Lesa and Manuel Garcia Herranz: Enabling Digital Health by Automatic Classification of Short Messages. Short paper to appear in ACM Digital Health 2016.

(Pardon my French ;-)

Don't worry, I'm not going to embark in a tirade against religion. While I do believe, as Voltaire remarked, that “those who have the power to make you absurd have the power to make you unjust,” this is beyond religion.

We have been relentlessly led to believe, for decades now, that somehow ideologies do not have a place in politics anymore. Instead, all we should pursuit is a rational approach to practical problems.

Among the stupid things we believe, this is probably the most stupid one.

A political ideology is simply a collection of ideas that is more or less comprehensive, in the sense of covering different aspects of our social life. Indeed, each one of us lives in a society that is essentially kept together and driven by a political ideology, something we fail to notice until the ideology changes or we experience a different society. It is like the smell of the city we live in, something we don't notice until we come back from a long trip.

The ideologies we live in are made of many ideas, some of them good, some of them bad. Saying that political ideologies are dead is just an attempt to convince us that we shouldn't revise the ideas that drive our particular society at a particular moment, because they have somehow proven to be correct.

I don’t think that is the case. We have been wrong about lots of things in the past, even in the recent past, and most likely we continue being wrong about a whole lot of other things, right now.


LOTR: The Two Towers (2002)

The ideologies that we live in are very important in politics because they determine which proposals for change are considered seriously, and which are summarily dismissed. In the political arena, ideologies determine who is reasonable and who is insane.

By accepting, against all rational thought, that the particular political ideology in which we live is somehow optimal, we have decided that we don't want to hear anything that challenges it. This fossilizes deeply held but ultimately stupid beliefs, including:

  1. That the greed of others is good for us.
  2. That the next war will make us safe.
  3. That politics and politicians are bad.
  4. That something is always going to save us, -or-

    That it is best to sit and wait for a collapse.

Abandoning these and other stupid beliefs won't solve everything: one can take great ideas and great intentions, and do something awful with them. However, many of the worst decisions we've collectively taken during 2015 (and accepting them passively is part of that) ultimately can be traced to some of these bad ideas we haven't been able to revise.

We believe the greed of others is good for us

This is a faulty generalization from the observation that, under specific circumstances, specific types of greed can create specific social benefits. For instance, when Adam Smith coined his famous “invisible hand” phrase, he was referring specifically to the preference of investors for domestic investment instead of foreign one. Instead, we have discarded the contexts in which greed might be good, and take this as if it where some universal law of nature.

Believing that the greed of others is good for us has allowed entire industrial sectors to capture the regulators that are suppose to keep them from harming us (and themselves!). Greed in our political system means economic interests determine what changes and what stays the same, while the interests of citizens have little influence.

Even worse, the defense of unrestricted greed has tied it to concepts that have nothing to do with it. For instance, placing limits on greed doesn't mean we don't respect private property. Instead, it means we want the property of the poor to be protected, not only the property of the rich. To me this has nothing to do with ending capitalism. But maybe I'm wrong, and, as Slavoj Žižek has repeatedly warned, I find it easier to imagine the end of the world than the end of capitalism.

We believe the next war will make us safe

In the US, heavy furnitures such as unstable bookshelves and large TV sets crush to death about 30 people every year, many of them children. Preventing their deaths would not require expensive military campaigns, yet the "war on terror" seems to be exactly what the US public think it needs in order to feel safe.


V from Vendetta (2005)

In France, after the Paris attacks the president asks and obtains “special powers” than he now wants to make permanent through constitutional amendments. War mongering leaders in Europe and America play with fear and swear they will protect us … if we just give them a little more power.

They say they will protect us by starting and winning a war. At the end of the war, there will be celebrations with music and fireworks, and everyone who was against us will surrender and quietly go home while the credits roll. They will promise never to harm us again. The world will be at peace.

Except it won't. It never has.

Peace is difficult to achieve, and killing people is a quite cinematic but fairly ineffective way of progressing towards that goal. Peace has many pre-conditions including an effective government, low levels of corruption, a sound business environment, acceptance of the rights of others, and high levels of human capital. Too many of us believe safety will be achieved through more wars; too few of us are thinking on how to create peace.

We believe politics and politicians are bad

Contrary to popular belief, our politicians don't fall from the sky. They are born among us, and while they often come from wealthy families, in the end it is we who vote for them. Do we have the ones we deserve? Maybe.

Politics is a way of distributing power and it plays a role in revising ideologies. Neither politics nor politicians are inherently bad. They are part of a process.

The problem is the process we have right now creates strong incentives for politicians to focus on two things, none of which is good for us. First, politicians have incentives to publicly and vocally support one or two policies favored by undecided voters, not majorities. Second, politicians have incentives to privately and quietly support whatever favors the elites who can help them get elected in their next campaign.

These things won't change overnight, but leaving democracy to its own devices will hardly make anything to improve it, unless …

We believe something will save us in the end -or-
Waiting for a collapse is the best

Mainstream left- and right-wing politicians rarely agree on something. When they agree on something, it is usually along the lines of some corporate interest. In the case of global warming, the left believes it is a serious problem, the right believes it is not, and the consensus is that it is a serious problem but we should do nothing about it.

The underlying belief is that something will save us in the end. Yes, the temperature will rise a few degrees, some polar bears will have to get a job in the circus, and a few small islands will be lost to the rising seas, nothing to worry about. Someone will invent an app or a machine that will make greenhouses go away, or perhaps others will change their habits so we won't need to.

This is indeed stupid; particularly considering the price tag of this stupidity might be astronomically high.

Some believe this is part of an impending collapse that ultimately will be good for us. I am not speaking about judgment day as, among others, extremist Christians and Muslims expect to happen any time soon.

Instead, I am speaking of a “rational” strategy, which is as follows. First, we completely refrain from political participation so that governments become increasingly illegitimate. Second, as we withdraw to the fringes of the system, we let a few people control most of the resources and take all the decisions. Third, we allow conditions in the planet to deteriorate to the point where things are unbearable and people start to die. Fourth, we chop a few heads, rename the months of the year, and start over.

Great plan—where do I click to support it?

It is easy to forget that we're a primitive society

As we celebrate progress, it is easy to forget the obvious fact that we live in a fairly primitive society. We stand divided, in more than one sense. Most of us can only communicate with a fraction of our fellow humans. None of us has ever left the close vicinity of our home.

“Maybe you earn less than your parents because you don’t have ideologies” said journalist Antonio Baños to masses of unemployed and underemployed voters in an interview.

We do have ideologies, plenty of them, the problem is that we don't recognize them as such, we take them as given, we tiptoe around them, we refuse to question them. We should understand they are opinions, not facts, and—to quote Voltaire once more—“opinions have caused more ills than the plague or earthquakes on this little globe of ours.”

Pages

Subscribe to ChaTo (Carlos Castillo) RSS