This post outlines evidence strongly suggesting Bahraini ISPs Batelco and Zain have been deliberately and covertly disabling fixed-line and mobile Internet services every night in Duraz, an area in which there are ongoing protests.
1. Executive Summary
Since 23 June 2016, residents of Duraz and neighboring villages along Budaiya Road been complaining of Internet disruptions,1 including apparent deactivation of mobile Internet services by Bahrain’s only mobile providers, Batelco, VIVA, and Zain, and unusable Batelco fixed-line DSL services from approximately 7:00PM to 1:00AM every night. Duraz has been the focal point of ongoing protests since 20 June 2016, when the government revoked the citizenship of Sheikh Isa Qassim, a popular Shia religious leader and resident of Duraz.
Until now, there has been no technical investigation of the reported Internet disruption to determine its scope, nature, or likely cause. As Internet and mobile services are typically oversubscribed, some level of disruption is to be expected when a large number of customers attempt to use the services at the same time, such as may be the case during a protest.
In this report, we investigate the disruption to Batelco’s fixed-line services, as well as the disruption to Batelco’s and Zain’s mobile data service in Duraz. Our experiments show that between 7PM and 1AM, certain 3G and 4G cell towers belonging to Batelco and Zain appear to be turned off, and 2G cells broadcast notifications to phones indicating that mobile data services are not supported. Our experiments also reveal the presence of a device on Batelco’s Internet backbone that disrupts certain Internet traffic to and from Duraz between 7PM and 1AM, while leaving other traffic undisrupted. Based on these findings, we conclude that Batelco and Zain are likely deliberately disrupting both fixed-line and mobile data services in Duraz. Furthermore, Batelco appears to have changed its disruption on July 12, the same day as the first major Bahraini media coverage of the Internet problems in Duraz. The new disruption appears to be more precisely targeted at users in Duraz, while affecting fewer users outside of Duraz.
Given that the disruptions are coordinated at roughly the same time across different ISPs, it is possible that the disruptions are a result of a Service Restriction Order (SRO) from the Bahrain Government, in relation to the protests. International organizations including the United Nations, as well as industry bodies including the GSM Association, have condemned this type of Internet interference by governments.
This section provides background on the protests, as well as Internet disruptions in general.
2.1. Protests in Duraz
Duraz is home to one of the central spiritual leaders of Bahrain’s Shia community, Sheikh Isa Qassim. On 20 June 2016, his citizenship was revoked by Bahrain’s Interior Ministry, rendering him stateless.2 Hundreds of people have been peacefully protesting this decision with sit-ins around his house in Duraz that take place in the evening.
Figure 1: Protests in Duraz around the home of religious leader Sheikh Isa Qassim (Source: @NseejNews).3
In response, the government has placed the community on lockdown, blocking all but two roads that lead into to the village; the two remaining entrances are controlled by police checkpoints. Reports indicate that police allow only residents of the village (as identified by the address shown on their national ID cards) to cross the checkpoints.4
2.2. Measuring Internet Shutdowns
Large-scale Internet shutdowns and disruptions typically result in a large decrease in Internet traffic originating from a country. Such disruptions can be noticed quickly by major content providers, or organizations that monitor Internet connectivity. However, small-scale disruptions, like the one affecting Duraz, may be much harder to detect and attribute in the same way. Access Now first reported on the Internet issues in Duraz, citing sources on the ground, but noted that they could not confirm the Internet disruption, given an absence of technical evidence.5
Figure 2: Location of the village of Duraz in the North of Bahrain (Source: Google Maps).
Bahrain’s ISPs have acknowledged issues with Internet connectivity in Duraz. On 12 July, Al-Wasat newspaper reported that ISPs said the disruption was a technical issue, and they were working on a fix.6 The most specific public response by an ISP to the disruption seems to be a 29 June public Twitter message from the official7 @BatelcoSupport account, which told one individual that there was a “general issue in Budhiya Exch,” presumably a reference to a telecom exchange point in Budaiya.
Figure 3: Tweet from @BatelcoSupport explaining Internet disruption to concerned customer.8
2.3. Condemnations of Internet Interference
International organizations and trade bodies have urged against government interference with Internet connectivity. On 30 June 2016, the UN Human Rights Council adopted a resolution9 condemning state measures that disrupt access to information online. The resolution contained the following operative clause:
Condemns unequivocally measures to intentionally prevent or disrupt access to or dissemination of information online in violation of international human rights law and calls on all States to refrain from and cease such measures;
The GSM Association, an industry body representing mobile phone providers, recently articulated a policy position regarding government restrictions of Internet access. The policy encourages governments to minimize the functional and temporal scope of any orders to telecom companies to disrupt services, and encourages both governments and telecom companies to communicate with subscribers about the restriction in a transparent fashion.10
3. A Tale of Two Disruptions (to Batelco Fixed-Line Connections)
This section describes our technical measurements of the disruption to fixed-line Internet connections in Duraz. We began measuring the first disruption on 27 June. The first Internet disruption continued every night, until 1AM on 12 July 2016. The night of 12 July saw a new disruption at a different location on Batelco’s network, which affected fewer users, and continues nightly as of the date of this report’s publication. The disruption may have changed in response to the first major media coverage of the Internet problems in Duraz by popular independent newspaper Al-Wasat, on 12 July.
Both disruptions result in astronomical levels of loss and/or latency in some subscribers’ Internet traffic, rendering subscribers’ connections unusable for most online activities between around 7PM and 1AM every night.
3.1. First Disruption (D1)
We began our investigation of Internet disruptions in Duraz after hearing reports that fixed-line Batelco connections in Duraz and neighboring villages along Budaiya Road were “very slow” for Internet browsing, while some other applications that used small packet sizes were unaffected.
We began regular hourly scans of Batelco’s entire IP address space to see if we could identify hosts that dropped larger packets more often than smaller packets. For each hourly scan, we first scanned Batelco’s IP address space with zmap’s ICMP echo scan.11 Roughly 20% of Batelco’s 111,360 IP addresses12 responded to our ICMP probes. We probed each responsive IP with 50 packets for each of 3 different packet sizes (128 bytes, 528 bytes, and 1028 bytes), at a rate of roughly 10 packets per minute per host. We classified an IP as Disrupted if less than 50% of 128B packets were dropped, more than 50% of 1028B packets were dropped, and the packet dropping rate for 528B packets was strictly in between the rates for 128B and 1028B packets. We show results for six days in Figure 4: the first disruption ended at 1AM on July 11th. On July 12th, a new disruption started, which did not result in larger packets being dropped more frequently than smaller packets. We describe the new disruption in Section 3.2.
Figure 4: Charting Disruption D1 Over Time.13
On average, roughly 12% of IP addresses that responded to our ICMP probes were disrupted every night by D1, including IP addresses outside of Duraz (see Section 3.1.2).
3.1.1. Localizing the disruption
We next sought to localize the disruption in Batelco’s network. We first performed IP traceroutes from a host in Bahrain that experienced the nightly disruptions, in order to get a general map of various paths on Batelco’s fixed-line network. We generated the map in Figure 5, which is how we think the network looks from the perspective of a subscriber.
Figure 5: Map of Batelco’s Network by IP Traceroute. Last octet of some IP addresses redacted.
Since we identified two different IP next-hops from the subscriber, we sought to test both paths to see if both were disrupted. We ran measurements from our experiment machine in the United States, and a disrupted host in Bahrain. We had the host in Bahrain ping 220.127.116.11, and we had the host in the United States ping the host in Bahrain. According to traceroutes, these packets transited through different TTL=2 routers from the perspective of the host in Bahrain. However, the loss rate over time for these two paths appeared to be closely related (Figure 6).
Figure 6: Measurements on two paths from a disrupted host in Bahrain.
Given the similarity in loss rate over time, we suspect that D1 occurred on a link in between TTL=1 and TTL=2 on both paths (Figure 7). Such a link would need to involve a device that is a Layer 1 or Layer 2 device, or another device that does not affect the IP TTL fields of subscriber packets.
Figure 7: Suspected location of disruption D1. Paths listed in green appeared to have no disruption present. The disruption is not necessarily on a link directly connected to the subscriber’s equipment, but occurs on some link between the subscriber and both first IP hops.
3.1.2. Corporate customers affected by D1
We were able to identify at least two corporate customers whose IP addresses appear to have been affected by disruption D1: Ithmaar Bank,14 and Al-Wasat Newspaper.15 We identified corporate customers by checking which static IP addresses were affected by D1 (by checking if their reverse DNS included the word “static”). We found 13 static IP addresses were affected, and used PassiveTotal16 to see if any domain names were mapped to the IP addresses (Table 1).
|IP Addresses Disrupted||Current and Historical Domain Names (from PassiveTotal)||Company that Owns the Domain Names|
Note that none of these IP addresses in Table 1 is currently used to serve either company’s main web site; the disruption probably instead affected other infrastructure services belonging to each company, or Internet connectivity from their corporate premises. In the case of Al-Wasat, the disruption affected what appears to be their backup mail server (mail2.alwasatnews.com).
In Figure 8, we plot the location of Al-Wasat newspaper in relation to Duraz. Given that Al-Wasat newspaper is located outside of Duraz, and experienced a similar disruption to that affecting subscribers in Duraz, we suspect that D1 may have affected some customers outside of Duraz.
Figure 8: Location of Al-Wasat Newspaper Offices in Relation to Duraz.
3.1.3. Characterizing disruption D1
We sought to precisely characterize the relation between loss rate and packet size for disrupted IPs in Bahrain. To this end, we tested six disrupted IP addresses arbitrarily selected from our scans for disrupted hosts. We tested the six IPs over the course of one day’s disruption by sending ICMP packets from our measurement machine to the disrupted IPs, ranging in size from 32 to 1496 bytes (in 8 byte increments) including ICMP and IP headers. We sent packets in pairs so the size of each pair was constant (i.e., (32B,1496B), (40B,1488B), etc). Our results appear in Figure 9.
Figure 9: Different Drop Rates for Different IPs.
We found different loss rate curves for different IPs. Unfortunately, we were not able to complete enough measurements to precisely attribute D1 to a specific mechanism.
3.1.4. End of disruption D1
On 12 July 2016, Bahrain’s popular independent Al-Wasat newspaper provided the first local mass media report focusing on the Internet disruption.17 That evening, D1 did not occur as scheduled. Instead, a second disruption, D2, replaced D1.
3.2. Second Disruption (D2)
On the evening of 12 July 2016, we noticed that IP addresses in Duraz appeared to be inaccessible from our measurement machine in the USA. Upon further inspection, we realized that a very small amount of traffic was delivered, with astronomical latency, and no discernable difference in drop probability between large and small packets. As D1 was biased against larger packets, and did not result in a latency increase, we hypothesized that a different disruption was affecting Duraz, and sought to locate it.
Figure 10: Various Measurements on July 17.
We conducted a series of measurements on July 17, shown in Figure 10. For measurements 1 and 2, we sent packets from a disrupted host in Bahrain to 18.104.22.168, and to our experiment machine in the USA. We noted that there was an average of 99% loss on both of these round-trips. However, as measurement 5 illustrates, there was no significant loss when pinging 22.214.171.124, the hop before .241. Thus, we hypothesize that the disruption occurs on the link between 126.96.36.199 an 188.8.131.52. Note that this is the same link where we previously identified that Batelco was dropping Telegram traffic.18
We also noted that the disruption on this link appeared to affect only traffic destined for or originating from certain IP addresses. For instance, pings we sent from the USA to 184.108.40.206 were not disrupted (measurement 6), despite apparently flowing over the disrupted link. However, pings we sent from the USA to Duraz, with TTL values set to terminate at 220.127.116.11, were disrupted (measurement 4). The disruption to measurement 4 appeared to be inconsistent with a round-trip disruption (measurements 1 and 2), and instead appeared to be more consistent with a one-way disruption (measurement 3). This is consistent with our hypothesis about the disruption being targeted based on source or destination IP: the ICMP packets we sent to 18.104.22.168 were destined for an IP address in Duraz, whereas the ICMP TTL Exceeded responses sent by 22.214.171.124 to our experiment machine in the USA did not mention a disrupted IP address.
We performed another measurement, where we asked an individual in Duraz to restart their router during the disruption. The individual restarted the router and was assigned a new IP address. Their old IP address immediately stopped experiencing disruption (TTL-limited pings we sent to the old address no longer experienced disruption), and the new IP address they were assigned was disrupted. It is possible that the selection of which IP addresses to disrupt may be based on the street address where the subscriber assigned the IP address has subscribed to DSL service. This might imply the system performing the disruption is integrated with the Batelco system that maps subscriber information to IP addresses (e.g., perhaps through a RADIUS tap).
Figure 11: Suspected location of disruption D2. Paths listed in green appeared to have no disruption present.
3.2.1. Comparing D2 with D1
Disruption D2 appears to start exactly at 7PM, and end exactly at 1AM each night, whereas the start and end times of Disruption D1 appeared to vary several minutes, night-to-night. We were also able to determine that D2 affects far fewer subscribers than D1. To measure the scope of D2, we undertook regular hourly scans of Batelco’s entire IP address space. Every hour, we scanned Batelco’s IP address space 100 times using zmap’s ICMP echo scan, while running tcpdump. After the scanning was completed, we examined our tcpdump output to find IP addresses that responded to less than 20 of the 100 probes during the hour, and whose ICMP echo responses we received at least 3 seconds after we sent the corresponding ICMP echo request. We present results for one day in Figure 12. Our results show that D2 affects about 0.5% of subscribers we could measure, whereas D1 affected about 12%. Additionally, the Al-Wasat Newspaper and Ithmaar Bank IPs we identified as affected by D1 are unaffected by D2.
Figure 12: Measuring Disruption D2 from July 27-28.
We suspect that D2 is targeted more precisely at the residents of Duraz than D1 was.
4. Analysis of Disruption to Mobile Networks
Bahrain’s three mobile providers, Viva, Zain, and Batelco, have separate GSM (2G), UMTS (3G), and LTE (4G) networks. Each provider and network (2G, 3G, 4G) has its own set of cells, identified by CID (Cell IDs). A BTS (Base Transceiver Station, i.e., mobile phone tower) may serve several cells. A mobile phone accesses phone and Internet services through a cell. As network conditions change, or the phone moves, the phone may switch to a different cell or network.
We examined radio traces of mobile phone activity for 3G-capable phones connected to Zain’s and Batelco’s network in Duraz. We noted no disruption to Viva when we tried to use mobile data services in Duraz, so we did not perform more extensive testing on Viva.
The phones were stationary during each test. Before the disruption, the phones were connected to 3G. Around 7PM, the phones automatically switched to 2G. We noted that our phones received a large number of “Immediate Assignment” messages when the phones switched to 2G, perhaps indicating that other subscribers’ phones were switching to 2G as well. “Immediate Assignment” messages are broadcast by a cell to all phones on a common channel as part of the IMSI attach procedure when a handset associates with the 2G network.19
Phone calls and SMS text messages functioned normally when we tested them during the disruption, though no type of mobile data service was available. Around 1AM, the phones switched back to 3G.
4.1. 3G and 4G Cells Shut Down?
We set our Zain test phone to “3G only” mode, to force it to stay on the 3G band even if it could not associate. In order for a phone to associate with a 3G UMTS cell, the phone must first receive a UMTS MIB (Master Information Block) message from the cell. MIB messages are periodically broadcast by a cell, and give the phone information it needs in order to read subsequent SIB (System Information Block) messages transmitted by that cell; those subsequent messages give the phone the information it needs to associate with the cell. Our measurements during the disruption show that our phone did not receive any UMTS MIB messages advertising Zain’s network. The lack of UMTS MIB messages for Zain would prevent Zain subscribers’ phones from knowing that the Zain 3G network was available in the area, so the phones would not try to connect. However, our Zain test phone did receive MIB messages for Viva’s and Batelco’s networks.
One type of SIB message is the SIB3 message, which carries information identifying the cell that sent it. During the disruption, our “3G only” Zain phone received SIB3 messages for one Batelco cell (CID 712077XX),20 and six Viva cells (five CID 1366XX,21 and one CID 1349XX). At around 1AM when the disruption ended, our “3G only” Zain phone began receiving MIB messages for Zain’s network, and began receiving associated SIB3 messages for two Zain cells (CID 14098XX).
We similarly tested a Batelco 3G phone set to “3G only” mode. During the disruption, we saw SIB3 messages from two Batelco cells (one CID 712077XX — the same Batelco cell observed by our Zain phone — and another Batelco CID 712056XX). When the disruption was not in effect, our Batelco phone received SIB3 messages from three different Batelco cells (CID 712061XX). We summarize our results in Table 2.
|Disruption in Effect||No Disruption|
We suspect that Zain and Batelco are disabling certain 3G cells during the disruption, which results in the absence of SIB3 messages for our phone’s preferred cells on these networks (712061XX for Batelco, 14098XX for Zain). We suspect that the Batelco cells we do receive SIB3 messages from during the disruption (712056XX and 712077XX) are located further away from our test phone than the preferred cells.
Given that Zain and Batelco LTE phones we tested also failed to connect to the LTE network during the disruption, we assume that Zain and Batelco are also disabling certain LTE cells during the disruption.
4.2. GPRS/EDGE Disabled on 2G Cells
Recall that during the disruption, the 2G networks of Batelco and Zain are available. However, as we did not notice any 2G mobile data service available (GPRS/EDGE), we examined further to determine the cause.
A 2G cell can indicate to phones whether or not it supports mobile data services (GPRS/EDGE) in a SI3 (Service Information 3) or SI4 (Service Information 4) message by setting the “GPRS Indicator” bit.22 On both Batelco and Zain, we observed several cells “turn off” GPRS/EDGE during the disruption: they sent SI3 messages with their GPRS Indicator bit on before and after the disruption, but sent SI3 messages with their GPRS Indicator bit off during the disruption. For Zain we observed ten 2G cells “turn off” mobile data services during the disruption, and for Batelco we observed seven such cells.
Figure 13: An example of SI3 messages received by our phone from the same cell before (left) and during (right) the disruption, showing the GPRS Indicator is present before the disruption, but not present during the disruption.
During the disruption, we observed infrequent SI3 messages from one cell on Batelco, and one cell on Zain, with the GPRS Indicator bit sent. Since we received these messages less frequently than SI3 messages for other cells, we suspect that these GPRS-enabled cells might be located further away, perhaps outside of Duraz.
We suspect that what we have observed is the result of Batelco and Zain deliberately disabling GPRS/EDGE functionality on their 2G cells in Duraz, while leaving such functionality enabled on neighboring cells outside of Duraz.
5. Arrest of Anonymous Twitter User: a Related Case?
The Ministry of Interior reported on 26 July that it had arrested an individual accused of using a fake social media account impersonating a telecom company to spread incorrect information “opposite to reality.” The prosecutor ordered the individual jailed for 15 days, pending a court hearing on the charge of broadcasting false news.23
The Twitter account in question may have been @zainbh, which appeared to be a non-official account for Zain (their official account is @ZainBahrain). Tweets from @zainbh (Figure 14) were widely circulated on WhatsApp on 24 and 25 July, and the account was suspended shortly thereafter.
Figure 14: Screenshots of the unofficial @zainbh account.
The tweets, addressed to Zain customers in Duraz and the surrounding areas, apologized for the disruption of the Internet, and said that in response to the “illegal assembly” in Duraz, the Ministry of Interior issued a request to cut the Internet connectivity in Duraz. The tweets also said that Zain was not responsible for the problems, and that affected customers should contact the Ministry of Interior for compensation.
According to our research, the @zainbh account was registered in March 2010, and appeared to tweet once in 2014, until it started tweeting during the Internet disruption.
We find that from around 7PM until 1AM every night, two Bahraini ISPs, Batelco and Zain, are likely disabling their 4G and 3G networks in Duraz, and turning off mobile data services (GPRS/EDGE) on their 2G mobile networks. While phone calls and SMS text messages are possible during the disruption, no type of mobile data service is available on Batelco and Zain.
We also find that between exactly 7PM and 1AM every night, Batelco is deliberately introducing astronomical levels of loss and latency into fixed-line Internet connections in Duraz. Before July 12, Batelco implemented a disruption that affected around 12% of subscribers we could measure, including corporate customers outside of Duraz. After the disruption received major media coverage in Bahrain’s Al-Wasat newspaper on July 12, Batelco appears to have switched to a more precisely targeted disruption, which affects around 0.5% of subscribers. This second disruption occurs on what appears to be a backbone link in Batelco’s network. As traffic flows across this backbone link, a device disrupts this traffic only if it is going to or coming from an IP address currently assigned to a targeted subscriber.
Given that the disruption appears to be coordinated between two different ISPs (Batelco and Zain) at roughly the same time every night (7PM until 1AM), we suspect that the disruption may be the result of a Service Restriction Order (SRO) from the Bahrain Government. We call on Batelco, Zain, and the Bahrain Government to publicize any SRO in effect, as urged by the GSM Association’s policy position on SROs.
We also call on Batelco, Zain, VIVA, and the Government to end the disruption, provide compensation to those affected, and publicize any SRO.
6.2. A New Type of Information Control
Large-scale disconnection or censorship of Internet services that affect entire regions or countries are well-studied phenomena. When governments employ such tactics for political ends, they are easily detected, and rebuke from civil society and the international community is swift.
Batelco’s landline Internet disruption appears to be targeted at individual subscriber accounts that Batelco believes to be associated with Duraz. This use of targeted Internet disruption for political ends represents a new form of information control: the muzzling of individuals — in this case potential protesters — that the government does not wish to hear. Muzzling allows a government to prevent individuals from speaking when the world needs to hear them the most, in a way that is hard for researchers to detect and attribute, all while avoiding the political consequences of large-scale Internet disruption.
- See http://bna.bh/portal/en/news/733106. The Government of Bahrain amended its citizenship laws in July of 2014 so that they may revoke the nationality of anyone who does not fufil their “duty of loyalty,” see https://www.hrw.org/news/2014/08/21/bahrain-citizenship-rights-stripped-away.
- The gap in the graph at 15:00 on 7/11 was caused by an error in executing zmap.
- We redact the last two digits for most Cell IDs we give in this report.
- When we write this, we mean that the Cell IDs differ only in their last two digits.