proc_17_108_analysis_01_level_0_manual_tagging.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Summary\n",
    "\n",
    "This notebook allows us to manually flag clustered items by signature string which breaks down the problem a little bit more. We can do this iteratively with EDA--send the results of the manual labeling back to EDA and try to find more clusters with the manually flagged clusters removed.\n",
    "\n",
    "As stated in the previous notebook, the point here isn't to exhaustively catch all clusters with 'signature strings', we just want to cut the problem down to size so we can apply HDBSCAN and systematically 'cluster' the rest of the text. Use the term 'cluster' loosely here -- see summary of next notebook  (proc_17_108_analysis_jeffykao_HDBScan_Clustering)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "pd.options.display.max_colwidth = 500"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loaded in 2955186 entries aggregating 22078910.0 total comments.\n",
      "CPU times: user 20 s, sys: 1.28 s, total: 21.3 s\n",
      "Wall time: 22.7 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "data = pd.read_csv('proc_17_108_unique_comments_text_dupe_count.csv')\n",
    "print(f\"Loaded in {len(data)} entries aggregating {data['dupe_count'].sum()} total comments.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2955182 Text entries aggregating 22078856.0 total comments.\n"
     ]
    }
   ],
   "source": [
    "# drop na's\n",
    "data.dropna(axis=0, how='any', subset=['text_data'], inplace=True)\n",
    "data.reset_index(inplace=True, drop=True)\n",
    "\n",
    "# initially unclustered\n",
    "data['level_0_cluster'] = -1\n",
    "\n",
    "print(f\"{len(data)} Text entries aggregating {data['dupe_count'].sum()} total comments.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "### slight detour ###\n",
    "# curious about the top duplicated comments (without any clustering)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dupe_count</th>\n",
       "      <th>text_data</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1096832.0</td>\n",
       "      <td>In 2015, Chairman Tom Wheelers Federal Communications Commission (FCC) imposed restrictive Title II, utility-style regulations under the guise of an open internet. Not only have these regulations inhibited innovation in the internet ecosystem, they hurt taxpayers and consumers by expanding the regulatory reach of the FCC and limiting investment in internet infrastructure. We cannot allow this revolutionary tool to be bogged down with excessive government interference.\\n \\nIt is past time ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>817622.0</td>\n",
       "      <td>The unprecedented regulatory power the Obama Administration imposed on the internet is smothering innovation, damaging the American economy and obstructing job creation.\\n\\nI urge the Federal Communications Commission to end the bureaucratic regulatory overreach of the internet known as Title II and restore the bipartisan light-touch regulatory consensus that enabled the internet to flourish for more than 20 years.\\n\\nThe plan currently under consideration at the FCC to repeal Obama's Title ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>550000.0</td>\n",
       "      <td>The current FCC regulatory scheme known as \"Title II\" represents an unprecedented increase in government control over the internet. Such over-regulation is hurting our economy and suffocating innovation.\\n \\nI support Chairman Pai's plan to return to a commonsense regulatory framework that allows for the internet to grow without useless government interference. The internet has flourished for decades without the heavy hand of government over-regulation.  Its time we return to what works.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>545551.0</td>\n",
       "      <td>Before leaving office, the Obama Administration rammed through a massive scheme that gave the federal government broad regulatory control over the internet. That misguided policy decision is threatening innovation and hurting broadband investment in one of the largest and most important sectors of the U.S. economy.\\n \\nI support the Federal Communications Commission's decision to roll back Title II and allow for free market principles to guide our digital economy.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>468436.0</td>\n",
       "      <td>The Obama-era FCC regulations known as \"Title II\" enable the federal government to exert an extraordinary and unnecessary amount of regulatory control over the internet. This bureaucratic overreach impedes innovation, stifles investment and continues to create economic uncertainty for one of the largest sectors of the U.S. economy.\\n \\nI support Chairman Pai's proposal to roll back Title II and restore the sensible regulatory framework that enjoyed broad bipartisan consensus and enabled the ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>451706.0</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked FaceTime, T...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>422042.0</td>\n",
       "      <td>As a concerned taxpayer and consumer, I am writing to urge the FCC to set the internet free and remove the inappropriate, unnecessary and overly vast regulations currently holding back the full potential of the internet. Due to the grip of the utility-style regulations imposed under the previous Commission, taxpayers have been put at risk, the threat of new fees on consumer bills still looms large, investment in internet infrastructure has not realized its full potential, innovations have go...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>379644.0</td>\n",
       "      <td>Obama's Federal Communications Commission (FCC) forced regulations on the internet that put the government, and unaccountable bureaucrats, in control. These rules have cost taxpayers, slowed down broadband infrastructure investment, and hindered competition and choice for Americans. The time to remove the regulatory stranglehold on the internet is NOW.\\n\\nI urge the taxpayer-funded FCC to undo the terrible regulatory burdens that ex-FCC Chairman Tom Wheeler imposed on the internet. After 20 ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>370000.0</td>\n",
       "      <td>Before leaving office, the Obama Administration rammed through a massive scheme that gave the federal government broad regulatory control over the internet. That misguided policy decision is threatening innovation and hurting broadband investment in one of the largest and most important sectors of the U.S. economy.\\n \\nI support the Federal Communications Commissions decision to roll back Title II and allow for free market principles to guide our digital economy.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>336066.0</td>\n",
       "      <td>I strongly oppose Chairman Pai&amp;apos;s proposal to reverse net neutrality protections because a free and open internet is vital for our democracy, for our businesses, and for our daily lives. It would give giant internet companies the power to prioritize what we read, watch, and explore online. I won&amp;apos;t stand for it. It&amp;apos;s about my right to be heard and my right to hear others. I submit my public comment to oppose Chairman Pai&amp;apos;s proposal to reverse net neutrality protections.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>307468.0</td>\n",
       "      <td>The current FCC regulatory scheme known as \"Title II\" represents an unprecedented increase in government control over the internet. Such over-regulation is hurting our economy and suffocating innovation.\\n \\nI support Chairman Pai's plan to return to a commonsense regulatory framework that allows for the internet to grow without useless government interference. The internet has flourished for decades without the heavy hand of government over-regulation.  It's time we return to what works.\\n</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>246371.0</td>\n",
       "      <td>In 2015, President Obama's FCC passed rules treating the Internet as a government regulated public utility for the first time in history. Those pushing hardest for the new rules were Silicon Valley monopolies like Google and leftist globalists like George Soros.\\n\\nGoogle in fact visited the White House more than 427 times during the Obama years. Leftist foundations like Soros' Open Society and the Ford Foundation spent almost $200 million pushing for the rule change.\\n\\nNow we know why. In ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>232844.0</td>\n",
       "      <td>In 2015, Chairman Tom Wheelers Federal Communications Commission (FCC) imposed restrictive Title II, utility-style regulations under the guise of an open internet. Not only have these regulations inhibited innovation in the internet ecosystem, they hurt taxpayers and consumers by expanding the regulatory reach of the FCC and limiting investment in internet infrastructure. We cannot allow this revolutionary tool to be bogged down with excessive government interference.\\n \\nIt is past time ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>232710.0</td>\n",
       "      <td>On July 12 is the Protect Net Neutrality Day of Action! This would be the day to protect our freedom of internet and Fight to escape the control of our life by the ISPs - the corporations we all despise. Do not let them get away with this, fight For Our Future as if our life depend on it!</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>225264.0</td>\n",
       "      <td>Don't kill net neutrality. We deserve a free and open Internet with strong Title II rules. This will ensure that the flow of data is determined by the interests of Internet users</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>217704.0</td>\n",
       "      <td>The Open Internet rules (net neutrality) are extremely important to me. I don't want the ISP to have the power to block websites, slow them down, give some sites advantage on others, or split the internet into fast lanes for companies that pay and slow lanes for the rest. Don't allow the Trump administration control the media streaming and then make only his NEWS On July 12, we should show the world how we Fight for The Future. I hope all media outlets join us. Thank you</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>216144.0</td>\n",
       "      <td>The Open Internet rules (net neutrality) are extremely important to me. I don't want the ISP to have the power to block websites, slow them down, give some sites advantage on others, or split the internet into fast lanes for companies that pay and slow lanes for the rest. Don't allow corporate America to control our life. Our internet is our access to knowledge and bridge between cultures and people. On July 12, we should show the FCC and the Trump administration how we Fight for The Future.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>205778.0</td>\n",
       "      <td>In 2015, wealthy leftist billionaires and powerful Silicon Valley monopolies took the internet out of the hands of the people and placed it firmly under the thumb of the federal the government, monopolies like Google and global billionaires like George Soros. Not surprisingly, today ObamaÕs new Internet gatekeepers are censoring our viewpoints, banning our online activities and silencing dissenting voices. As Google Chairman Eric Schmidt admitted, ÒWeÕre not arguing for censorship, weÕre arg...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>167929.0</td>\n",
       "      <td>Cable and phone companies provide access to the internet. They're telecommunications carriers. They do not (and should not) have the right to censor or slow down my speech and my access to online content. When I use my broadband service, I decide who I communicate with and what information I transmit. I want the FCC to retain the ability to stop my internet service provider from interfering with my communications choices. The courts have already told the FCC that to do this, ISPs must remain...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>137549.0</td>\n",
       "      <td>Before leaving office, the Obama Administration rammed through a massive scheme that gave the federal government broad regulatory control over the internet. That misguided policy decision is threatening innovation and hurting broadband investment in one of the largest and most important sectors of the U.S. economy.\\n \\nI support the Federal Communications CommissionÍs decision to roll back Title II and allow for free market principles to guide our digital economy.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>122772.0</td>\n",
       "      <td>As a concerned taxpayer and consumer, I am writing to urge the FCC to set the internet free and remove the inappropriate, unnecessary and overly vast regulations currently holding back the full potential of the internet. Due to the grip of the utility-style regulations imposed under the previous Commission, taxpayers have been put at risk, the threat of new fees on consumer bills still looms large, investment in internet infrastructure has not realized its full potential, innovations have go...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>109425.0</td>\n",
       "      <td>Please save the internet from the corporations. Tom Wheeler was right. Let the new neutrality stand.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>109419.0</td>\n",
       "      <td>Dear Ladies and Gentlemen. Net neutrality is a serious topic. Please do not implement weakend regulations. TO keep it short and save my breath: All possible negative scenarios were discussed in the past. More than that: some of them were getting reality and thus the discussion started back in 2010/ So please keep it as it is and do something useful.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>109160.0</td>\n",
       "      <td>make sure net neutrality does not dissapear. It is the only thing saving the internet at this moment. If it is removed many webservices will be at risk.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>109122.0</td>\n",
       "      <td>We should not leave the ability for small companies to compete with large online businesses up to the whims of our internet providers. It is out responsibility to defend our right to free market competition. IF THENEWSEARCHENGINE  is better than Google, Google's wealth should not strike down the new engine if our internet providers choose not to be benevolent. Save net neutrality.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>108960.0</td>\n",
       "      <td>Please Save Net Neutrality and Title ll rules. As a freelance translator I rely on internet spped to do my work and corporations would interfere with my searches if given the change. Do the right thing and keep the needed regulations. Thank you.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>108861.0</td>\n",
       "      <td>Save net neutrality! I am worried that the protections that are in place will be weakened if we change the way they are enforced. I do not support losing any protections, and I do not trust big companies to look out for my best interests. That's YOUR JOB!</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>108771.0</td>\n",
       "      <td>We need net neutralityto continue. A free and open internet is the single greatest technology of our time, and control should not be at the mercy of corporations.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>108502.0</td>\n",
       "      <td>Allowing broadband providers to throttle their service is unfair business practive. Weakening protections for consumers for the sake of big business is foolish and a countrer to the FCC's mandate. This proposal clearly gets rid of net neutrality. I am opposed to this. Save net neutrality and protect consumers.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>108319.0</td>\n",
       "      <td>Leave the net neutrality alone. Internet speed should not be for sale. There is nothing wrong with out current system save for the greedy corporations who wish to turn the internet into their own pay-to-play and a government willing to let them do it. You serve the American people not just Verizon, AT&amp;T, etc. IF you think the current protests are bad, try touching new neutrality. I will never understand why a nation who has come so far is hell bent on going back to the 18th century. No other...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955152</th>\n",
       "      <td>1.0</td>\n",
       "      <td>To the FCC:  I'm a voter worried about regulations on the Internet. I strongly advocate the commissioners to reverse The previous administration's scheme to control broadband. Citizens, not Washington, should be empowered to enjoy whichever applications we choose. The previous administration's scheme to control broadband is a exploitation  of net neutrality. It broke a hands-off approach that performed very successfully for two decades with bipartisan support.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955153</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Chairman Pai:  In the matter of the FCC regulations on the Internet. I request the commissioners to undo Tom Wheeler's power grab to take over broadband. Americans, rather than so-called experts, should be able to purchase which products we desire. Tom Wheeler's power grab to take over broadband is a distortion of the open Internet. It broke a market-based policy that worked remarkably smoothly for two decades with Republican and Democrat support.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955154</th>\n",
       "      <td>1.0</td>\n",
       "      <td>My comments re: the Obama takeover of the Internet. I want to implore Chairman Pai to repeal Obama's order to control the Internet. Americans, rather than so-called experts, ought to buy the products we want. Obama's order to control the Internet is a corruption of the open Internet. It undid a pro-consumer system that functioned remarkably successfully for two decades with nearly universal consensus.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955155</th>\n",
       "      <td>1.0</td>\n",
       "      <td>To whom it may concern:  Hi, I'd like to comment on Net neutrality. I implore the FCC to repeal Barack Obama's decision to regulate Internet access. Individual Americans, rather than big government, should be empowered to purchase whatever products they choose. Barack Obama's decision to regulate Internet access is a perversion of net neutrality. It undid a free-market framework that worked supremely successfully for a long time with bipartisan approval.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955156</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Dear Commissioners:  I am concerned about the Open Internet order. I would like to demand the Federal Communications Commission to reverse President Obama's plan to regulate Internet access. Americans, rather than big government, ought to select whichever applications we desire. President Obama's plan to regulate Internet access is a corruption of net neutrality. It broke a hands-off framework that performed fabulously well for a long time with bipartisan approval.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955157</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Mr Pai:  With respect to the future of the Internet. I strongly recommend the Federal Communications Commission to repeal Tom Wheeler's policy to control broadband. Individuals, not so-called experts, should enjoy which services we choose. Tom Wheeler's policy to control broadband is a distortion of the open Internet. It disrupted a market-based framework that performed exceptionally well for a long time with bipartisan consensus.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955158</th>\n",
       "      <td>1.0</td>\n",
       "      <td>To whom it may concern:  I'm contacting you about the FCC's Open Internet order. I'd like to implore the commission to reverse Obama's policy to control the web. Individual Americans, rather than Washington, deserve to select which applications they want. Obama's policy to control the web is a betrayal of net neutrality. It ended a free-market framework that worked supremely well for many years with bipartisan consensus.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955159</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Chairman Pai:  I would like to comment on Internet regulation and net neutrality. I want to implore the FCC to overturn The Obama/Wheeler scheme to regulate broadband. People like me, not unelected bureaucrats, should be empowered to buy whichever products we desire. The Obama/Wheeler scheme to regulate broadband is a perversion of net neutrality. It stopped a market-based framework that performed remarkably smoothly for decades with nearly universal approval.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955160</th>\n",
       "      <td>1.0</td>\n",
       "      <td>To whom it may concern:  In reference to net neutrality rules. I strongly recommend the commission to rescind Tom Wheeler's scheme to take over broadband. Internet users, as opposed to unelected bureaucrats, should be free to use whichever products they choose. Tom Wheeler's scheme to take over broadband is a perversion of net neutrality. It undid a hands-off approach that performed remarkably well for a long time with both parties' backing.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955161</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Dear Commissioners:  I am a voter worried about Internet Freedom. I'd like to request the Federal Communications Commission to repeal President Obama's order to control Internet access. Individual Americans, rather than Washington bureaucrats, should be free to select the products they desire. President Obama's order to control Internet access is a distortion of net neutrality. It disrupted a hands-off system that worked remarkably well for decades with both parties' backing.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955162</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Mr Pai:  I would like to comment on net neutrality. I'd like to implore Chairman Pai to reverse Tom Wheeler's order to control the web. Citizens, rather than so-called experts, ought to use the services they want. Tom Wheeler's order to control the web is a distortion of the open Internet. It stopped a pro-consumer policy that worked very, very well for a long time with both parties' support.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955163</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Chairman Pai:  I am a voter worried about Title II rules. I'd like to ask you to overturn The previous administration's decision to regulate the web. Individual citizens, rather than so-called experts, should be able to select which applications we desire. The previous administration's decision to regulate the web is a perversion of the open Internet. It broke a light-touch framework that performed fabulously successfully for many years with both parties' support.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955164</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Dear Chairman Pai,  With respect to Title 2 and net neutrality. I'd like to request the commission to undo Obama's plan to take over the web. Citizens, rather than big government, should be empowered to purchase which services we want. Obama's plan to take over the web is a corruption of net neutrality. It undid a free-market system that worked remarkably well for decades with Republican and Democrat backing.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955165</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Chairman Pai:  My comments re: Internet regulation and net neutrality. I'd like to encourage the Federal Communications Commission to overturn The Obama/Wheeler plan to control Internet access. Americans, not the FCC Enforcement Bureau, should select the applications we want. The Obama/Wheeler plan to control Internet access is a exploitation  of the open Internet. It disrupted a light-touch system that performed very, very successfully for two decades with broad bipartisan approval.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955166</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Mr Pai:  Hi, I'd like to comment on regulations on the Internet. I urge the commissioners to rescind The previous administration's power grab to regulate the Internet. People like me, rather than unelected bureaucrats, ought to use whichever applications they prefer. The previous administration's power grab to regulate the Internet is a betrayal of the open Internet. It stopped a free-market framework that worked very smoothly for two decades with both parties' backing.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955167</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Mr Pai:  I have concerns about Internet Freedom. I'd like to encourage the Federal Communications Commission to rescind Barack Obama's plan to regulate Internet access. Individual Americans, rather than Washington, ought to use which services we choose. Barack Obama's plan to regulate Internet access is a distortion of the open Internet. It broke a free-market policy that functioned very, very well for decades with bipartisan backing.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955168</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Chairman Pai:  I'd like to share my thoughts on an open Internet. I would like to urge the government to overturn Tom Wheeler's scheme to control Internet access. People like me, not Washington bureaucrats, should be free to enjoy whichever products they prefer. Tom Wheeler's scheme to control Internet access is a perversion of net neutrality. It undid a light-touch framework that functioned very successfully for decades with bipartisan consensus.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955169</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Dear FCC,  I'm very concerned about the Obama takeover of the Internet. I would like to suggest the Federal Communications Commission to reverse President Obama's policy to control broadband. Individual Americans, not the FCC Enforcement Bureau, ought to select whichever services they choose. President Obama's policy to control broadband is a exploitation  of the open Internet. It stopped a pro-consumer policy that performed very, very successfully for decades with broad bipartisan support.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955170</th>\n",
       "      <td>1.0</td>\n",
       "      <td>I'd like to share my thoughts on net neutrality and Title II. I'd like to ask the commission to rescind President Obama's order to take over broadband. Individual citizens, as opposed to the FCC, should be free to select the applications they desire. President Obama's order to take over broadband is a perversion of the open Internet. It disrupted a hands-off policy that functioned supremely successfully for two decades with broad bipartisan approval.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955171</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Dear Chairman Pai,  My comments re: Network Neutrality. I strongly ask the commission to repeal The Obama/Wheeler order to control the Internet. Individuals, rather than so-called experts, ought to use which products we desire. The Obama/Wheeler order to control the Internet is a distortion of the open Internet. It disrupted a market-based system that functioned very smoothly for a long time with nearly universal consensus.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955172</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Dear FCC,  I'm contacting you about network neutrality regulations. I strongly ask the commission to reverse President Obama's plan to take over Internet access. Individual Americans, not the FCC, deserve to buy whichever applications they prefer. President Obama's plan to take over Internet access is a distortion of the open Internet. It undid a light-touch system that worked fabulously smoothly for a long time with nearly universal approval.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955173</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Regarding Title 2 and net neutrality. I want to request Ajit Pai to overturn President Obama's decision to take over broadband. Citizens, not Washington bureaucrats, deserve to select the applications they choose. President Obama's decision to take over broadband is a exploitation  of the open Internet. It stopped a light-touch approach that functioned remarkably successfully for decades with broad bipartisan backing.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955174</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Dear Chairman Pai,  I'm a voter worried about the Obama takeover of the Internet. I want to advocate you to rescind President Obama's order to control Internet access. Internet users, as opposed to so-called experts, should select whatever products they desire. President Obama's order to control Internet access is a betrayal of net neutrality. It disrupted a market-based system that functioned supremely smoothly for decades with bipartisan support.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955175</th>\n",
       "      <td>1.0</td>\n",
       "      <td>To the Federal Communications Commission:  I would like to comment on the FCC's Open Internet order. I want to demand the commission to undo Barack Obama's power grab to take over broadband. Citizens, not Washington, should be empowered to select which applications they choose. Barack Obama's power grab to take over broadband is a distortion of net neutrality. It disrupted a pro-consumer policy that performed very, very successfully for two decades with nearly universal backing.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955176</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Mr Pai:  I want to give my opinion on net neutrality. I implore the commissioners to rescind President Obama's power grab to regulate the Internet. Individual citizens, as opposed to the FCC, deserve to select whichever applications we desire. President Obama's power grab to regulate the Internet is a distortion of net neutrality. It disrupted a light-touch framework that functioned supremely smoothly for many years with both parties' backing.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955177</th>\n",
       "      <td>1.0</td>\n",
       "      <td>To the Federal Communications Commission:  Hi, I'd like to comment on Net Neutrality and Title II. I'd like to request the commission to rescind Barack Obama's plan to regulate broadband. Citizens, not the FCC Enforcement Bureau, should be able to use whichever applications we prefer. Barack Obama's plan to regulate broadband is a distortion of the open Internet. It reversed a pro-consumer policy that worked supremely well for two decades with both parties' support.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955178</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Chairman Pai:  My comments re: Internet regulation. I suggest the commissioners to reverse Barack Obama's scheme to control the web. Internet users, rather than Washington bureaucrats, ought to select whichever services they choose. Barack Obama's scheme to control the web is a distortion of the open Internet. It undid a free-market policy that functioned very successfully for decades with broad bipartisan backing.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955179</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Dear Mr. Pai,  I'd like to share my thoughts on NET NEUTRALITY. I want to advocate Chairman Pai to overturn President Obama's scheme to regulate the Internet. Individual Americans, as opposed to big government, should be empowered to buy the services they prefer. President Obama's scheme to regulate the Internet is a perversion of the open Internet. It stopped a hands-off framework that performed very smoothly for decades with bipartisan consensus.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955180</th>\n",
       "      <td>1.0</td>\n",
       "      <td>FCC:  With respect to Internet regulation and net neutrality. I strongly ask the commissioners to repeal Obama's decision to take over the web. Individual citizens, not Washington bureaucrats, should be able to buy whatever products we want. Obama's decision to take over the web is a distortion of the open Internet. It disrupted a light-touch system that functioned fabulously smoothly for many years with broad bipartisan support.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2955181</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Please preserve net neutrality and title 2. You will crush small businesses if you allow this insanity. No politicians want to be known as the ones who ended small business.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2955182 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         dupe_count  \\\n",
       "0         1096832.0   \n",
       "1          817622.0   \n",
       "2          550000.0   \n",
       "3          545551.0   \n",
       "4          468436.0   \n",
       "5          451706.0   \n",
       "6          422042.0   \n",
       "7          379644.0   \n",
       "8          370000.0   \n",
       "9          336066.0   \n",
       "10         307468.0   \n",
       "11         246371.0   \n",
       "12         232844.0   \n",
       "13         232710.0   \n",
       "14         225264.0   \n",
       "15         217704.0   \n",
       "16         216144.0   \n",
       "17         205778.0   \n",
       "18         167929.0   \n",
       "19         137549.0   \n",
       "20         122772.0   \n",
       "21         109425.0   \n",
       "22         109419.0   \n",
       "23         109160.0   \n",
       "24         109122.0   \n",
       "25         108960.0   \n",
       "26         108861.0   \n",
       "27         108771.0   \n",
       "28         108502.0   \n",
       "29         108319.0   \n",
       "...             ...   \n",
       "2955152         1.0   \n",
       "2955153         1.0   \n",
       "2955154         1.0   \n",
       "2955155         1.0   \n",
       "2955156         1.0   \n",
       "2955157         1.0   \n",
       "2955158         1.0   \n",
       "2955159         1.0   \n",
       "2955160         1.0   \n",
       "2955161         1.0   \n",
       "2955162         1.0   \n",
       "2955163         1.0   \n",
       "2955164         1.0   \n",
       "2955165         1.0   \n",
       "2955166         1.0   \n",
       "2955167         1.0   \n",
       "2955168         1.0   \n",
       "2955169         1.0   \n",
       "2955170         1.0   \n",
       "2955171         1.0   \n",
       "2955172         1.0   \n",
       "2955173         1.0   \n",
       "2955174         1.0   \n",
       "2955175         1.0   \n",
       "2955176         1.0   \n",
       "2955177         1.0   \n",
       "2955178         1.0   \n",
       "2955179         1.0   \n",
       "2955180         1.0   \n",
       "2955181         1.0   \n",
       "\n",
       "                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   text_data  \n",
       "0        In 2015, Chairman Tom Wheelers Federal Communications Commission (FCC) imposed restrictive Title II, utility-style regulations under the guise of an open internet. Not only have these regulations inhibited innovation in the internet ecosystem, they hurt taxpayers and consumers by expanding the regulatory reach of the FCC and limiting investment in internet infrastructure. We cannot allow this revolutionary tool to be bogged down with excessive government interference.\\n \\nIt is past time ...  \n",
       "1        The unprecedented regulatory power the Obama Administration imposed on the internet is smothering innovation, damaging the American economy and obstructing job creation.\\n\\nI urge the Federal Communications Commission to end the bureaucratic regulatory overreach of the internet known as Title II and restore the bipartisan light-touch regulatory consensus that enabled the internet to flourish for more than 20 years.\\n\\nThe plan currently under consideration at the FCC to repeal Obama's Title ...  \n",
       "2              The current FCC regulatory scheme known as \"Title II\" represents an unprecedented increase in government control over the internet. Such over-regulation is hurting our economy and suffocating innovation.\\n \\nI support Chairman Pai's plan to return to a commonsense regulatory framework that allows for the internet to grow without useless government interference. The internet has flourished for decades without the heavy hand of government over-regulation.  Its time we return to what works.  \n",
       "3                                       Before leaving office, the Obama Administration rammed through a massive scheme that gave the federal government broad regulatory control over the internet. That misguided policy decision is threatening innovation and hurting broadband investment in one of the largest and most important sectors of the U.S. economy.\\n \\nI support the Federal Communications Commission's decision to roll back Title II and allow for free market principles to guide our digital economy.  \n",
       "4        The Obama-era FCC regulations known as \"Title II\" enable the federal government to exert an extraordinary and unnecessary amount of regulatory control over the internet. This bureaucratic overreach impedes innovation, stifles investment and continues to create economic uncertainty for one of the largest sectors of the U.S. economy.\\n \\nI support Chairman Pai's proposal to roll back Title II and restore the sensible regulatory framework that enjoyed broad bipartisan consensus and enabled the ...  \n",
       "5        The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked FaceTime, T...  \n",
       "6        As a concerned taxpayer and consumer, I am writing to urge the FCC to set the internet free and remove the inappropriate, unnecessary and overly vast regulations currently holding back the full potential of the internet. Due to the grip of the utility-style regulations imposed under the previous Commission, taxpayers have been put at risk, the threat of new fees on consumer bills still looms large, investment in internet infrastructure has not realized its full potential, innovations have go...  \n",
       "7        Obama's Federal Communications Commission (FCC) forced regulations on the internet that put the government, and unaccountable bureaucrats, in control. These rules have cost taxpayers, slowed down broadband infrastructure investment, and hindered competition and choice for Americans. The time to remove the regulatory stranglehold on the internet is NOW.\\n\\nI urge the taxpayer-funded FCC to undo the terrible regulatory burdens that ex-FCC Chairman Tom Wheeler imposed on the internet. After 20 ...  \n",
       "8                                       Before leaving office, the Obama Administration rammed through a massive scheme that gave the federal government broad regulatory control over the internet. That misguided policy decision is threatening innovation and hurting broadband investment in one of the largest and most important sectors of the U.S. economy.\\n \\nI support the Federal Communications Commissions decision to roll back Title II and allow for free market principles to guide our digital economy.  \n",
       "9               I strongly oppose Chairman Pai&apos;s proposal to reverse net neutrality protections because a free and open internet is vital for our democracy, for our businesses, and for our daily lives. It would give giant internet companies the power to prioritize what we read, watch, and explore online. I won&apos;t stand for it. It&apos;s about my right to be heard and my right to hear others. I submit my public comment to oppose Chairman Pai&apos;s proposal to reverse net neutrality protections.  \n",
       "10           The current FCC regulatory scheme known as \"Title II\" represents an unprecedented increase in government control over the internet. Such over-regulation is hurting our economy and suffocating innovation.\\n \\nI support Chairman Pai's plan to return to a commonsense regulatory framework that allows for the internet to grow without useless government interference. The internet has flourished for decades without the heavy hand of government over-regulation.  It's time we return to what works.\\n  \n",
       "11       In 2015, President Obama's FCC passed rules treating the Internet as a government regulated public utility for the first time in history. Those pushing hardest for the new rules were Silicon Valley monopolies like Google and leftist globalists like George Soros.\\n\\nGoogle in fact visited the White House more than 427 times during the Obama years. Leftist foundations like Soros' Open Society and the Ford Foundation spent almost $200 million pushing for the rule change.\\n\\nNow we know why. In ...  \n",
       "12       In 2015, Chairman Tom Wheelers Federal Communications Commission (FCC) imposed restrictive Title II, utility-style regulations under the guise of an open internet. Not only have these regulations inhibited innovation in the internet ecosystem, they hurt taxpayers and consumers by expanding the regulatory reach of the FCC and limiting investment in internet infrastructure. We cannot allow this revolutionary tool to be bogged down with excessive government interference.\\n \\nIt is past time ...  \n",
       "13                                                                                                                                                                                                                         On July 12 is the Protect Net Neutrality Day of Action! This would be the day to protect our freedom of internet and Fight to escape the control of our life by the ISPs - the corporations we all despise. Do not let them get away with this, fight For Our Future as if our life depend on it!  \n",
       "14                                                                                                                                                                                                                                                                                                                                        Don't kill net neutrality. We deserve a free and open Internet with strong Title II rules. This will ensure that the flow of data is determined by the interests of Internet users  \n",
       "15                               The Open Internet rules (net neutrality) are extremely important to me. I don't want the ISP to have the power to block websites, slow them down, give some sites advantage on others, or split the internet into fast lanes for companies that pay and slow lanes for the rest. Don't allow the Trump administration control the media streaming and then make only his NEWS On July 12, we should show the world how we Fight for The Future. I hope all media outlets join us. Thank you  \n",
       "16          The Open Internet rules (net neutrality) are extremely important to me. I don't want the ISP to have the power to block websites, slow them down, give some sites advantage on others, or split the internet into fast lanes for companies that pay and slow lanes for the rest. Don't allow corporate America to control our life. Our internet is our access to knowledge and bridge between cultures and people. On July 12, we should show the FCC and the Trump administration how we Fight for The Future.  \n",
       "17       In 2015, wealthy leftist billionaires and powerful Silicon Valley monopolies took the internet out of the hands of the people and placed it firmly under the thumb of the federal the government, monopolies like Google and global billionaires like George Soros. Not surprisingly, today ObamaÕs new Internet gatekeepers are censoring our viewpoints, banning our online activities and silencing dissenting voices. As Google Chairman Eric Schmidt admitted, ÒWeÕre not arguing for censorship, weÕre arg...  \n",
       "18       Cable and phone companies provide access to the internet. They're telecommunications carriers. They do not (and should not) have the right to censor or slow down my speech and my access to online content. When I use my broadband service, I decide who I communicate with and what information I transmit. I want the FCC to retain the ability to stop my internet service provider from interfering with my communications choices. The courts have already told the FCC that to do this, ISPs must remain...  \n",
       "19                                      Before leaving office, the Obama Administration rammed through a massive scheme that gave the federal government broad regulatory control over the internet. That misguided policy decision is threatening innovation and hurting broadband investment in one of the largest and most important sectors of the U.S. economy.\\n \\nI support the Federal Communications CommissionÍs decision to roll back Title II and allow for free market principles to guide our digital economy.  \n",
       "20       As a concerned taxpayer and consumer, I am writing to urge the FCC to set the internet free and remove the inappropriate, unnecessary and overly vast regulations currently holding back the full potential of the internet. Due to the grip of the utility-style regulations imposed under the previous Commission, taxpayers have been put at risk, the threat of new fees on consumer bills still looms large, investment in internet infrastructure has not realized its full potential, innovations have go...  \n",
       "21                                                                                                                                                                                                                                                                                                                                                                                                                      Please save the internet from the corporations. Tom Wheeler was right. Let the new neutrality stand.  \n",
       "22                                                                                                                                                           Dear Ladies and Gentlemen. Net neutrality is a serious topic. Please do not implement weakend regulations. TO keep it short and save my breath: All possible negative scenarios were discussed in the past. More than that: some of them were getting reality and thus the discussion started back in 2010/ So please keep it as it is and do something useful.  \n",
       "23                                                                                                                                                                                                                                                                                                                                                                  make sure net neutrality does not dissapear. It is the only thing saving the internet at this moment. If it is removed many webservices will be at risk.  \n",
       "24                                                                                                                           We should not leave the ability for small companies to compete with large online businesses up to the whims of our internet providers. It is out responsibility to defend our right to free market competition. IF THENEWSEARCHENGINE  is better than Google, Google's wealth should not strike down the new engine if our internet providers choose not to be benevolent. Save net neutrality.  \n",
       "25                                                                                                                                                                                                                                                                     Please Save Net Neutrality and Title ll rules. As a freelance translator I rely on internet spped to do my work and corporations would interfere with my searches if given the change. Do the right thing and keep the needed regulations. Thank you.  \n",
       "26                                                                                                                                                                                                                                                           Save net neutrality! I am worried that the protections that are in place will be weakened if we change the way they are enforced. I do not support losing any protections, and I do not trust big companies to look out for my best interests. That's YOUR JOB!  \n",
       "27                                                                                                                                                                                                                                                                                                                                                        We need net neutralityto continue. A free and open internet is the single greatest technology of our time, and control should not be at the mercy of corporations.  \n",
       "28                                                                                                                                                                                                   Allowing broadband providers to throttle their service is unfair business practive. Weakening protections for consumers for the sake of big business is foolish and a countrer to the FCC's mandate. This proposal clearly gets rid of net neutrality. I am opposed to this. Save net neutrality and protect consumers.  \n",
       "29       Leave the net neutrality alone. Internet speed should not be for sale. There is nothing wrong with out current system save for the greedy corporations who wish to turn the internet into their own pay-to-play and a government willing to let them do it. You serve the American people not just Verizon, AT&T, etc. IF you think the current protests are bad, try touching new neutrality. I will never understand why a nation who has come so far is hell bent on going back to the 18th century. No other...  \n",
       "...                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      ...  \n",
       "2955152                                     To the FCC:  I'm a voter worried about regulations on the Internet. I strongly advocate the commissioners to reverse The previous administration's scheme to control broadband. Citizens, not Washington, should be empowered to enjoy whichever applications we choose. The previous administration's scheme to control broadband is a exploitation  of net neutrality. It broke a hands-off approach that performed very successfully for two decades with bipartisan support.  \n",
       "2955153                                                  Chairman Pai:  In the matter of the FCC regulations on the Internet. I request the commissioners to undo Tom Wheeler's power grab to take over broadband. Americans, rather than so-called experts, should be able to purchase which products we desire. Tom Wheeler's power grab to take over broadband is a distortion of the open Internet. It broke a market-based policy that worked remarkably smoothly for two decades with Republican and Democrat support.  \n",
       "2955154                                                                                                 My comments re: the Obama takeover of the Internet. I want to implore Chairman Pai to repeal Obama's order to control the Internet. Americans, rather than so-called experts, ought to buy the products we want. Obama's order to control the Internet is a corruption of the open Internet. It undid a pro-consumer system that functioned remarkably successfully for two decades with nearly universal consensus.  \n",
       "2955155                                           To whom it may concern:  Hi, I'd like to comment on Net neutrality. I implore the FCC to repeal Barack Obama's decision to regulate Internet access. Individual Americans, rather than big government, should be empowered to purchase whatever products they choose. Barack Obama's decision to regulate Internet access is a perversion of net neutrality. It undid a free-market framework that worked supremely successfully for a long time with bipartisan approval.  \n",
       "2955156                                Dear Commissioners:  I am concerned about the Open Internet order. I would like to demand the Federal Communications Commission to reverse President Obama's plan to regulate Internet access. Americans, rather than big government, ought to select whichever applications we desire. President Obama's plan to regulate Internet access is a corruption of net neutrality. It broke a hands-off framework that performed fabulously well for a long time with bipartisan approval.  \n",
       "2955157                                                                   Mr Pai:  With respect to the future of the Internet. I strongly recommend the Federal Communications Commission to repeal Tom Wheeler's policy to control broadband. Individuals, not so-called experts, should enjoy which services we choose. Tom Wheeler's policy to control broadband is a distortion of the open Internet. It disrupted a market-based framework that performed exceptionally well for a long time with bipartisan consensus.  \n",
       "2955158                                                                             To whom it may concern:  I'm contacting you about the FCC's Open Internet order. I'd like to implore the commission to reverse Obama's policy to control the web. Individual Americans, rather than Washington, deserve to select which applications they want. Obama's policy to control the web is a betrayal of net neutrality. It ended a free-market framework that worked supremely well for many years with bipartisan consensus.  \n",
       "2955159                                     Chairman Pai:  I would like to comment on Internet regulation and net neutrality. I want to implore the FCC to overturn The Obama/Wheeler scheme to regulate broadband. People like me, not unelected bureaucrats, should be empowered to buy whichever products we desire. The Obama/Wheeler scheme to regulate broadband is a perversion of net neutrality. It stopped a market-based framework that performed remarkably smoothly for decades with nearly universal approval.  \n",
       "2955160                                                        To whom it may concern:  In reference to net neutrality rules. I strongly recommend the commission to rescind Tom Wheeler's scheme to take over broadband. Internet users, as opposed to unelected bureaucrats, should be free to use whichever products they choose. Tom Wheeler's scheme to take over broadband is a perversion of net neutrality. It undid a hands-off approach that performed remarkably well for a long time with both parties' backing.  \n",
       "2955161                     Dear Commissioners:  I am a voter worried about Internet Freedom. I'd like to request the Federal Communications Commission to repeal President Obama's order to control Internet access. Individual Americans, rather than Washington bureaucrats, should be free to select the products they desire. President Obama's order to control Internet access is a distortion of net neutrality. It disrupted a hands-off system that worked remarkably well for decades with both parties' backing.  \n",
       "2955162                                                                                                          Mr Pai:  I would like to comment on net neutrality. I'd like to implore Chairman Pai to reverse Tom Wheeler's order to control the web. Citizens, rather than so-called experts, ought to use the services they want. Tom Wheeler's order to control the web is a distortion of the open Internet. It stopped a pro-consumer policy that worked very, very well for a long time with both parties' support.  \n",
       "2955163                                 Chairman Pai:  I am a voter worried about Title II rules. I'd like to ask you to overturn The previous administration's decision to regulate the web. Individual citizens, rather than so-called experts, should be able to select which applications we desire. The previous administration's decision to regulate the web is a perversion of the open Internet. It broke a light-touch framework that performed fabulously successfully for many years with both parties' support.  \n",
       "2955164                                                                                         Dear Chairman Pai,  With respect to Title 2 and net neutrality. I'd like to request the commission to undo Obama's plan to take over the web. Citizens, rather than big government, should be empowered to purchase which services we want. Obama's plan to take over the web is a corruption of net neutrality. It undid a free-market system that worked remarkably well for decades with Republican and Democrat backing.  \n",
       "2955165             Chairman Pai:  My comments re: Internet regulation and net neutrality. I'd like to encourage the Federal Communications Commission to overturn The Obama/Wheeler plan to control Internet access. Americans, not the FCC Enforcement Bureau, should select the applications we want. The Obama/Wheeler plan to control Internet access is a exploitation  of the open Internet. It disrupted a light-touch system that performed very, very successfully for two decades with broad bipartisan approval.  \n",
       "2955166                           Mr Pai:  Hi, I'd like to comment on regulations on the Internet. I urge the commissioners to rescind The previous administration's power grab to regulate the Internet. People like me, rather than unelected bureaucrats, ought to use whichever applications they prefer. The previous administration's power grab to regulate the Internet is a betrayal of the open Internet. It stopped a free-market framework that worked very smoothly for two decades with both parties' backing.  \n",
       "2955167                                                               Mr Pai:  I have concerns about Internet Freedom. I'd like to encourage the Federal Communications Commission to rescind Barack Obama's plan to regulate Internet access. Individual Americans, rather than Washington, ought to use which services we choose. Barack Obama's plan to regulate Internet access is a distortion of the open Internet. It broke a free-market policy that functioned very, very well for decades with bipartisan backing.  \n",
       "2955168                                                  Chairman Pai:  I'd like to share my thoughts on an open Internet. I would like to urge the government to overturn Tom Wheeler's scheme to control Internet access. People like me, not Washington bureaucrats, should be free to enjoy whichever products they prefer. Tom Wheeler's scheme to control Internet access is a perversion of net neutrality. It undid a light-touch framework that functioned very successfully for decades with bipartisan consensus.  \n",
       "2955169      Dear FCC,  I'm very concerned about the Obama takeover of the Internet. I would like to suggest the Federal Communications Commission to reverse President Obama's policy to control broadband. Individual Americans, not the FCC Enforcement Bureau, ought to select whichever services they choose. President Obama's policy to control broadband is a exploitation  of the open Internet. It stopped a pro-consumer policy that performed very, very successfully for decades with broad bipartisan support.  \n",
       "2955170                                               I'd like to share my thoughts on net neutrality and Title II. I'd like to ask the commission to rescind President Obama's order to take over broadband. Individual citizens, as opposed to the FCC, should be free to select the applications they desire. President Obama's order to take over broadband is a perversion of the open Internet. It disrupted a hands-off policy that functioned supremely successfully for two decades with broad bipartisan approval.  \n",
       "2955171                                                                          Dear Chairman Pai,  My comments re: Network Neutrality. I strongly ask the commission to repeal The Obama/Wheeler order to control the Internet. Individuals, rather than so-called experts, ought to use which products we desire. The Obama/Wheeler order to control the Internet is a distortion of the open Internet. It disrupted a market-based system that functioned very smoothly for a long time with nearly universal consensus.  \n",
       "2955172                                                      Dear FCC,  I'm contacting you about network neutrality regulations. I strongly ask the commission to reverse President Obama's plan to take over Internet access. Individual Americans, not the FCC, deserve to buy whichever applications they prefer. President Obama's plan to take over Internet access is a distortion of the open Internet. It undid a light-touch system that worked fabulously smoothly for a long time with nearly universal approval.  \n",
       "2955173                                                                                Regarding Title 2 and net neutrality. I want to request Ajit Pai to overturn President Obama's decision to take over broadband. Citizens, not Washington bureaucrats, deserve to select the applications they choose. President Obama's decision to take over broadband is a exploitation  of the open Internet. It stopped a light-touch approach that functioned remarkably successfully for decades with broad bipartisan backing.  \n",
       "2955174                                                 Dear Chairman Pai,  I'm a voter worried about the Obama takeover of the Internet. I want to advocate you to rescind President Obama's order to control Internet access. Internet users, as opposed to so-called experts, should select whatever products they desire. President Obama's order to control Internet access is a betrayal of net neutrality. It disrupted a market-based system that functioned supremely smoothly for decades with bipartisan support.  \n",
       "2955175                  To the Federal Communications Commission:  I would like to comment on the FCC's Open Internet order. I want to demand the commission to undo Barack Obama's power grab to take over broadband. Citizens, not Washington, should be empowered to select which applications they choose. Barack Obama's power grab to take over broadband is a distortion of net neutrality. It disrupted a pro-consumer policy that performed very, very successfully for two decades with nearly universal backing.  \n",
       "2955176                                                      Mr Pai:  I want to give my opinion on net neutrality. I implore the commissioners to rescind President Obama's power grab to regulate the Internet. Individual citizens, as opposed to the FCC, deserve to select whichever applications we desire. President Obama's power grab to regulate the Internet is a distortion of net neutrality. It disrupted a light-touch framework that functioned supremely smoothly for many years with both parties' backing.  \n",
       "2955177                               To the Federal Communications Commission:  Hi, I'd like to comment on Net Neutrality and Title II. I'd like to request the commission to rescind Barack Obama's plan to regulate broadband. Citizens, not the FCC Enforcement Bureau, should be able to use whichever applications we prefer. Barack Obama's plan to regulate broadband is a distortion of the open Internet. It reversed a pro-consumer policy that worked supremely well for two decades with both parties' support.  \n",
       "2955178                                                                                   Chairman Pai:  My comments re: Internet regulation. I suggest the commissioners to reverse Barack Obama's scheme to control the web. Internet users, rather than Washington bureaucrats, ought to select whichever services they choose. Barack Obama's scheme to control the web is a distortion of the open Internet. It undid a free-market policy that functioned very successfully for decades with broad bipartisan backing.  \n",
       "2955179                                                 Dear Mr. Pai,  I'd like to share my thoughts on NET NEUTRALITY. I want to advocate Chairman Pai to overturn President Obama's scheme to regulate the Internet. Individual Americans, as opposed to big government, should be empowered to buy the services they prefer. President Obama's scheme to regulate the Internet is a perversion of the open Internet. It stopped a hands-off framework that performed very smoothly for decades with bipartisan consensus.  \n",
       "2955180                                                                    FCC:  With respect to Internet regulation and net neutrality. I strongly ask the commissioners to repeal Obama's decision to take over the web. Individual citizens, not Washington bureaucrats, should be able to buy whatever products we want. Obama's decision to take over the web is a distortion of the open Internet. It disrupted a light-touch system that functioned fabulously smoothly for many years with broad bipartisan support.  \n",
       "2955181                                                                                                                                                                                                                                                                                                                                        Please preserve net neutrality and title 2. You will crush small businesses if you allow this insanity. No politicians want to be known as the ones who ended small business.  \n",
       "\n",
       "[2955182 rows x 2 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "### look at most and least duplicated comments\n",
    "data_by_dupe = data.sort_values('dupe_count', ascending=False).reset_index(drop=True)\n",
    "data_by_dupe.loc[:,['dupe_count','text_data']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dupe_count</th>\n",
       "      <th>text_data</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1513933</th>\n",
       "      <td>1.0</td>\n",
       "      <td>I’m calling on the FCC to stand up for net neutrality and safeguard Title II protections. \\r\\n\\r\\nThe FCC should reject Chairman Ajit Pai’s proposal to hand the ISP monopolies like AT&amp;T, Verizon, and Comcast the authority to throttle whatever they please, stripping consumers of the necessary access and privacy rules we worked for and so recently won. \\r\\n\\r\\nI’m worried about creating a tiered Internet with “fast lanes” for certain sites or services because Think about the big picture here. ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>470814</th>\n",
       "      <td>2.0</td>\n",
       "      <td>Mr Pai:  I am concerned about Title 2 and net neutrality. I strongly implore the Federal Communications Commission to reverse President Obama's power grab to take over broadband. Americans, not the FCC, should be free to use whichever applications they desire. President Obama's power grab to take over broadband is a corruption of net neutrality. It disrupted a market-based policy that worked fabulously smoothly for two decades with bipartisan approval.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1805356</th>\n",
       "      <td>1.0</td>\n",
       "      <td>I am writing in support of Net Neutrality rules. I make my living as a web developer. I remember specifically a time back in the age of dial-up, when I was working for a small privately-owned Internet Service Provider. When Verizon decided to start offering their own dial-up access product,  the cost to maintain dial-up access points (i.e. virtual phone numbers people could use to dial in to access our internet service) rose dramatically. Perhaps it was coincidence, bue the bottom line is th...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2887808</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Dear Commissioners:  I want to give my opinion on net neutrality regulations. I would like to request the commission to repeal Obama's scheme to take over the Internet. Internet users, rather than the FCC, ought to purchase the products we want. Obama's scheme to take over the Internet is a perversion of net neutrality. It reversed a pro-consumer framework that functioned supremely smoothly for a long time with broad bipartisan consensus.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1997919</th>\n",
       "      <td>1.0</td>\n",
       "      <td>To the Federal Communications Commission:  I have concerns about the FCC regulations on the Internet. I advocate Ajit Pai to reverse President Obama's power grab to take over Internet access. People like me, as opposed to Washington bureaucrats, should use whichever services they desire. President Obama's power grab to take over Internet access is a exploitation  of the open Internet. It disrupted a light-touch framework that functioned very well for many years with Republican and Democrat b...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1782180</th>\n",
       "      <td>1.0</td>\n",
       "      <td>I am an American citizen and voter. I appreciate that the FCC is inviting public comments on chairman Ajit Pai's intent to switch the legal regulation of the FCC from Title 2 to Title 1. I am writing to express my strong support for net neutrality , and I oppose the proposed switch to Title 1. \\n\\nThank you for considering my comments.\\nMiranda Glasbergen</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1850861</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Mr Pai:  With respect to the FCC's so-called Open Internet order. I recommend you to overturn Barack Obama's scheme to regulate Internet access. Individuals, as opposed to big government, should be able to purchase whatever products we desire. Barack Obama's scheme to regulate Internet access is a distortion of net neutrality. It reversed a hands-off approach that worked supremely well for a long time with both parties' backing.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1982121</th>\n",
       "      <td>1.0</td>\n",
       "      <td>FCC commissioners,  I have concerns about NET NEUTRALITY. I strongly advocate the Federal Communications Commission to overturn The Obama/Wheeler power grab to control broadband. People like me, as opposed to Washington, should be empowered to buy the applications they prefer. The Obama/Wheeler power grab to control broadband is a perversion of net neutrality. It reversed a pro-consumer policy that performed remarkably smoothly for a long time with bipartisan approval.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>587944</th>\n",
       "      <td>2.0</td>\n",
       "      <td>FCC commissioners,  I'm concerned about NET NEUTRALITY. I would like to implore Chairman Pai to reverse Barack Obama's decision to regulate broadband. Individual citizens, as opposed to Washington, ought to enjoy whichever products we choose. Barack Obama's decision to regulate broadband is a betrayal of the open Internet. It disrupted a free-market policy that functioned very smoothly for many years with both parties' support.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2260691</th>\n",
       "      <td>1.0</td>\n",
       "      <td>I urge the FCC to continue upholding Obama era net neutrality regulations. I think these regulations are important to allowing government oversight to internet service providers, which otherwise might profit at the expense of access to the internet and international communication.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2603882</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Mr Pai:  I would like to comment on the FCC regulations on the Internet. I want to advocate you to undo The previous administration's scheme to take over broadband. Internet users, as opposed to big government, should be able to use which products they choose. The previous administration's scheme to take over broadband is a perversion of net neutrality. It broke a hands-off approach that performed very, very smoothly for many years with both parties' consensus.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2185523</th>\n",
       "      <td>1.0</td>\n",
       "      <td>The current Title II classification should be kept in place</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>742775</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Keep the open internet, they are trying to cash money out of blocking or making thing a lot slower</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2582610</th>\n",
       "      <td>1.0</td>\n",
       "      <td>To the Federal Communications Commission:  I'm a voter worried about internet regulations. I strongly demand you to rescind Barack Obama's order to take over the Internet. Americans, rather than Washington, should buy whichever products they choose. Barack Obama's order to take over the Internet is a betrayal of the open Internet. It undid a hands-off policy that performed very, very well for decades with Republican and Democrat consensus.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2289997</th>\n",
       "      <td>1.0</td>\n",
       "      <td>I strongly believe in the Title II classification required for oversight where Net Neutrality protections are concerned. This is necessary for a strong, free internet for every citizen.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1559919</th>\n",
       "      <td>1.0</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked FaceTime, T...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2579707</th>\n",
       "      <td>1.0</td>\n",
       "      <td>I'm a voter worried about the FCC regulations on the Internet. I ask Ajit Pai to rescind Tom Wheeler's policy to control broadband. Individual citizens, rather than Washington, deserve to use the applications we want. Tom Wheeler's policy to control broadband is a corruption of net neutrality. It disrupted a market-based framework that functioned exceptionally successfully for two decades with nearly universal backing.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2818845</th>\n",
       "      <td>1.0</td>\n",
       "      <td>To the Federal Communications Commission:  With respect to the FCC's Open Internet order. I strongly demand the government to overturn President Obama's power grab to control Internet access. People like me, rather than the FCC Enforcement Bureau, should be free to buy the applications they desire. President Obama's power grab to control Internet access is a betrayal of the open Internet. It reversed a market-based system that worked very, very successfully for decades with bipartisan consen...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2566708</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Dear Commissioners:  I want to give my opinion on Internet regulation and net neutrality. I want to urge you to repeal The Obama/Wheeler policy to regulate Internet access. Americans, not unelected bureaucrats, should be able to select whichever services they want. The Obama/Wheeler policy to regulate Internet access is a betrayal of net neutrality. It reversed a hands-off approach that worked very, very smoothly for a long time with both parties' support.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1634760</th>\n",
       "      <td>1.0</td>\n",
       "      <td>The government would best serve its citizens by repealing/removing regulations and subsidies.  Small government and few regulations is better and cheaper and allows for more innovation.\\n\\nObama's Internet takeover should be repealed.\\n</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1848023</th>\n",
       "      <td>1.0</td>\n",
       "      <td>As an Internet user, I'm asking the FCC to protect the net neutrality protections currently in place. \\n\\nThe FCC should throw out Chairman Ajit Pai's proposal to give the ISP monopolies like Comcast, AT&amp;T, and Verizon the authority to create Internet fast lanes, stripping Internet users of the meaningful access and privacy protections we fought for and just recently won. \\n\\nI'm concerned about ISPs being allowed to discriminate against certain types of data or websites, because users will ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1428928</th>\n",
       "      <td>1.0</td>\n",
       "      <td>PLEASE HELP SUSTAIN WHATEVER INTEGRITY IS LEFT IN THIS GREAT NATION.  THE INTERNET IS OUR LAST BASTION OF FREEDOM.  Do not let the lobbyists and corporate greed siphon the remaining dollars from our already gutted middle and lower class families through package upgrade fee implementations and other bandwidth throttling practices.  This is sheer madness.  It must be stopped.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2393200</th>\n",
       "      <td>1.0</td>\n",
       "      <td>To the FCC:  I'm concerned about Internet regulation and net neutrality. I strongly urge the Federal Communications Commission to reverse The Obama/Wheeler decision to take over broadband. Individuals, rather than Washington bureaucrats, should select which services we desire. The Obama/Wheeler decision to take over broadband is a betrayal of the open Internet. It reversed a market-based framework that performed supremely successfully for decades with both parties' backing.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2716380</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Chairman Pai:  I am concerned about restoring Internet freedom. I encourage the Federal Communications Commission to rescind Barack Obama's power grab to control the Internet. Americans, not unelected bureaucrats, deserve to buy whatever products we desire. Barack Obama's power grab to control the Internet is a distortion of net neutrality. It ended a light-touch approach that functioned very successfully for many years with nearly universal approval.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2400235</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Dear FCC,  I have thoughts on Title 2 and net neutrality. I want to demand you to undo The Obama/Wheeler policy to regulate Internet access. Americans, not big government, deserve to use the applications they prefer. The Obama/Wheeler policy to regulate Internet access is a exploitation  of the open Internet. It reversed a hands-off approach that functioned very, very well for a long time with Republican and Democrat approval.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2101661</th>\n",
       "      <td>1.0</td>\n",
       "      <td>I STRONGLY support \"Net Neutrality\" backed by Title II oversight of ISPs</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1406289</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Protect Net Neutrality, allow me, my family and my global neighbors to choose what we read and where we go on the net, instead of profit margins choosing.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2747741</th>\n",
       "      <td>1.0</td>\n",
       "      <td>To the FCC:  I'd like to share my thoughts on net neutrality and Title II. I want to recommend Chairman Pai to rescind The previous administration's policy to control the web. Americans, rather than the FCC, should purchase the applications we desire. The previous administration's policy to control the web is a perversion of the open Internet. It stopped a light-touch framework that worked exceptionally successfully for two decades with both parties' approval.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2308441</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Please save net neutrality. Protect our right to communication and we will be fighting for something that will have a lasting affect for centuries.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1320821</th>\n",
       "      <td>1.0</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked FaceTime, T...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2159377</th>\n",
       "      <td>1.0</td>\n",
       "      <td>I specifically support strong net neutrality backed by TITLE II oversight of ISP's. \\nKeeping Title 2 is the only way to ensure a fair and open internet for all Americans.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>428094</th>\n",
       "      <td>2.0</td>\n",
       "      <td>Dear Mr. Pai,  I'm a voter worried about net neutrality and Title II. I'd like to ask Ajit Pai to undo The previous administration's policy to regulate Internet access. Individual citizens, not Washington, should purchase which services we want. The previous administration's policy to regulate Internet access is a exploitation  of net neutrality. It undid a pro-consumer policy that worked remarkably well for a long time with nearly universal support.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2348222</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Dear Commissioners:  Hi, I'd like to comment on an open Internet. I want to request you to rescind The Obama/Wheeler order to control Internet access. Citizens, rather than the FCC, ought to buy which services they choose. The Obama/Wheeler order to control Internet access is a perversion of net neutrality. It broke a light-touch system that worked exceptionally smoothly for two decades with Republican and Democrat backing.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1176716</th>\n",
       "      <td>1.0</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked FaceTime, T...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1709464</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Dear Express Restoring Internet Freedom,\\n\\nI support Chairman Ajit PaiÕs rollback of ObamaÕs internet takeover. \\n\\nTitle II regulation gives too much power to the government over the future of the internet. \\n\\nIt has harmed innovation and investment. It also deters competition. \\n\\nMarkets and consumers should regulate the InternetÕs future, not government.\\n\\nRegards,\\nJanet Barrett\\n1289 Corbett Ln\\nMelbourne, FL 32904</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>537410</th>\n",
       "      <td>2.0</td>\n",
       "      <td>I want you to preserve Net Neutrality and maintain Title II for the benefit of all Americans.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>405464</th>\n",
       "      <td>2.0</td>\n",
       "      <td>Dear Chairman Pai,  I would like to comment on Internet Freedom. I would like to demand the government to repeal The previous administration's scheme to take over Internet access. Americans, as opposed to so-called experts, should be able to enjoy whatever services they choose. The previous administration's scheme to take over Internet access is a distortion of the open Internet. It broke a free-market approach that functioned exceptionally well for two decades with nearly universal backing.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1709184</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Dear Express Restoring Internet Freedom,\\n\\nI support Chairman Ajit PaiÕs rollback of ObamaÕs internet takeover. \\n\\nTitle II regulation gives too much power to the government over the future of the internet. \\n\\nIt has harmed innovation and investment. It also deters competition. \\n\\nMarkets and consumers should regulate the InternetÕs future, not government.\\n\\nRegards,\\nRose Oxley\\n151 State Route 33 Ste 252\\nManalapan, NJ 07726</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>656844</th>\n",
       "      <td>1.0</td>\n",
       "      <td>The internet is one of the true frontiers of the 21st century. It has become a landscape of diversity that is fundamental to modern society. One of the pillars that contributes to this diversity is neutrality of its operating basis. If Internet neutrality is compromised then it will the resources available will become increasingly top heavy with the most wealthy and powerful organizations. The availability to the smaller producer of content is what makes for such a diverse and enjoyable inte...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>974566</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Dear FCC,\\n\\nNet neutrality is essential to maintain the free market spirit of our economy. Removing net neutrality will have a devastating effect on the ability of startups to prosper and will stifle competition and innovation. Most areas of the country only have one available ISP, and therefore, they have near monopoly power and it is the responsibility of the government in this case to intervene because the barriers to entry in the market are too high. Please reconsider your rule changes ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>530297</th>\n",
       "      <td>2.0</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked FaceTime, T...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>317557</th>\n",
       "      <td>2.0</td>\n",
       "      <td>FCC commissioners,  I'm a voter worried about Internet Freedom. I want to ask Ajit Pai to repeal Barack Obama's power grab to take over the Internet. People like me, rather than the FCC, ought to enjoy whatever applications they choose. Barack Obama's power grab to take over the Internet is a exploitation  of the open Internet. It ended a free-market system that performed very, very smoothly for many years with Republican and Democrat backing.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>215980</th>\n",
       "      <td>2.0</td>\n",
       "      <td>To the FCC:  Regarding network neutrality regulations. I would like to ask the commission to overturn Obama's policy to regulate Internet access. Individual citizens, rather than big government, should select whatever products they choose. Obama's policy to regulate Internet access is a corruption of the open Internet. It ended a hands-off policy that functioned fabulously well for two decades with bipartisan backing.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2748401</th>\n",
       "      <td>1.0</td>\n",
       "      <td>FCC commissioners,  Hi, I'd like to comment on the FCC's so-called Open Internet order. I would like to ask the government to rescind Barack Obama's plan to regulate Internet access. Internet users, rather than the FCC Enforcement Bureau, should use whatever products they prefer. Barack Obama's plan to regulate Internet access is a distortion of the open Internet. It ended a pro-consumer approach that worked exceptionally well for decades with both parties' consensus.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2411230</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Dear FCC,  I'm contacting you about the FCC's so-called Open Internet order. I want to suggest the commissioners to reverse Tom Wheeler's plan to control broadband. Internet users, as opposed to Washington, ought to buy which products we desire. Tom Wheeler's plan to control broadband is a distortion of net neutrality. It undid a market-based approach that functioned remarkably successfully for two decades with nearly universal approval.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>241488</th>\n",
       "      <td>2.0</td>\n",
       "      <td>To the FCC:  I'm contacting you about the FCC regulations on the Internet. I would like to encourage the commission to repeal The previous administration's decision to control broadband. Individual citizens, as opposed to so-called experts, deserve to buy the applications we prefer. The previous administration's decision to control broadband is a distortion of the open Internet. It reversed a light-touch framework that functioned exceptionally smoothly for decades with nearly universal conse...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2388164</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Dear Chairman Pai,  I'd like to share my thoughts on Net neutrality. I'd like to recommend Chairman Pai to reverse Obama's policy to control the web. Citizens, not the FCC Enforcement Bureau, deserve to use which products they want. Obama's policy to control the web is a exploitation  of the open Internet. It reversed a free-market policy that worked exceptionally successfully for a long time with Republican and Democrat backing.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>497066</th>\n",
       "      <td>2.0</td>\n",
       "      <td>FCC:  With respect to restoring Internet freedom. I want to ask the commission to overturn President Obama's policy to take over broadband. Individuals, as opposed to Washington, ought to enjoy whichever applications we prefer. President Obama's policy to take over broadband is a betrayal of the open Internet. It undid a light-touch system that functioned exceptionally smoothly for a long time with bipartisan support.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1219678</th>\n",
       "      <td>1.0</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked FaceTime, T...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1059070</th>\n",
       "      <td>1.0</td>\n",
       "      <td>This is BAD! Very Bad!</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>689419</th>\n",
       "      <td>1.0</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked FaceTime, T...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>303003</th>\n",
       "      <td>2.0</td>\n",
       "      <td>Chairman Pai:  I'm concerned about an open Internet. I'd like to suggest you to overturn The previous administration's plan to control the Internet. People like me, rather than unelected bureaucrats, should buy whatever applications they desire. The previous administration's plan to control the Internet is a distortion of net neutrality. It reversed a light-touch policy that functioned very well for a long time with both parties' approval.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1599015</th>\n",
       "      <td>1.0</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\r\\n\\r\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\r\\n\\r\\nNow is not the time to let giant ISPs censor what we see and do online.\\r\\n\\r\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1460903</th>\n",
       "      <td>1.0</td>\n",
       "      <td>For integrity, truth, communication, education, growth personally and socially, for FREEDOM we NEED the OPEN INTERNET!!!!</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>786326</th>\n",
       "      <td>1.0</td>\n",
       "      <td>I’m calling on the FCC to stand up for net neutrality and safeguard Title II protections. \\r\\n\\r\\nThe FCC should reject Chairman Ajit Pai’s proposal to hand the government-subsidized telecom giants like Comcast, Verizon, and AT&amp;T the legal cover to engage in data discrimination, stripping consumers of the meaningful privacy and access protections we worked for and won just two years ago. \\r\\n\\r\\nI’m concerned about ISPs being allowed to discriminate against certain types of data or websites ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1584315</th>\n",
       "      <td>1.0</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\r\\n\\r\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\r\\n\\r\\nNow is not the time to let giant ISPs censor what we see and do online.\\r\\n\\r\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2026496</th>\n",
       "      <td>1.0</td>\n",
       "      <td>I’m worried that the protections that are in place will be weakened if changes are made to the way they’re enforced. I would support a new regulation style if it guarantees the same or better protections, but not if we lose any. I want all of the sites that I access on the web to be treated equally. Additionally I want my ISP to be fairly regulated so that no site or app can be blocked or pay for preferential treatment.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2751223</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Dear Mr. Pai,  I'm very worried about Net neutrality. I'd like to demand the commissioners to undo The previous administration's power grab to take over Internet access. Citizens, as opposed to Washington, should enjoy the services they want. The previous administration's power grab to take over Internet access is a betrayal of the open Internet. It disrupted a light-touch framework that worked very, very well for a long time with Republican and Democrat approval.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1128901</th>\n",
       "      <td>1.0</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\r\\n\\r\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\r\\n\\r\\nNow is not the time to let giant ISPs censor what we see and do online.\\r\\n\\r\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2596386</th>\n",
       "      <td>1.0</td>\n",
       "      <td>I'm contacting you about net neutrality and Title II. I strongly implore the Federal Communications Commission to undo Barack Obama's policy to regulate the Internet. People like me, rather than the FCC, should purchase the applications we choose. Barack Obama's policy to regulate the Internet is a betrayal of the open Internet. It stopped a light-touch framework that performed supremely well for decades with nearly universal approval.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         dupe_count  \\\n",
       "1513933         1.0   \n",
       "470814          2.0   \n",
       "1805356         1.0   \n",
       "2887808         1.0   \n",
       "1997919         1.0   \n",
       "1782180         1.0   \n",
       "1850861         1.0   \n",
       "1982121         1.0   \n",
       "587944          2.0   \n",
       "2260691         1.0   \n",
       "2603882         1.0   \n",
       "2185523         1.0   \n",
       "742775          1.0   \n",
       "2582610         1.0   \n",
       "2289997         1.0   \n",
       "1559919         1.0   \n",
       "2579707         1.0   \n",
       "2818845         1.0   \n",
       "2566708         1.0   \n",
       "1634760         1.0   \n",
       "1848023         1.0   \n",
       "1428928         1.0   \n",
       "2393200         1.0   \n",
       "2716380         1.0   \n",
       "2400235         1.0   \n",
       "2101661         1.0   \n",
       "1406289         1.0   \n",
       "2747741         1.0   \n",
       "2308441         1.0   \n",
       "1320821         1.0   \n",
       "2159377         1.0   \n",
       "428094          2.0   \n",
       "2348222         1.0   \n",
       "1176716         1.0   \n",
       "1709464         1.0   \n",
       "537410          2.0   \n",
       "405464          2.0   \n",
       "1709184         1.0   \n",
       "656844          1.0   \n",
       "974566          1.0   \n",
       "530297          2.0   \n",
       "317557          2.0   \n",
       "215980          2.0   \n",
       "2748401         1.0   \n",
       "2411230         1.0   \n",
       "241488          2.0   \n",
       "2388164         1.0   \n",
       "497066          2.0   \n",
       "1219678         1.0   \n",
       "1059070         1.0   \n",
       "689419          1.0   \n",
       "303003          2.0   \n",
       "1599015         1.0   \n",
       "1460903         1.0   \n",
       "786326          1.0   \n",
       "1584315         1.0   \n",
       "2026496         1.0   \n",
       "2751223         1.0   \n",
       "1128901         1.0   \n",
       "2596386         1.0   \n",
       "\n",
       "                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   text_data  \n",
       "1513933  I’m calling on the FCC to stand up for net neutrality and safeguard Title II protections. \\r\\n\\r\\nThe FCC should reject Chairman Ajit Pai’s proposal to hand the ISP monopolies like AT&T, Verizon, and Comcast the authority to throttle whatever they please, stripping consumers of the necessary access and privacy rules we worked for and so recently won. \\r\\n\\r\\nI’m worried about creating a tiered Internet with “fast lanes” for certain sites or services because Think about the big picture here. ...  \n",
       "470814                                              Mr Pai:  I am concerned about Title 2 and net neutrality. I strongly implore the Federal Communications Commission to reverse President Obama's power grab to take over broadband. Americans, not the FCC, should be free to use whichever applications they desire. President Obama's power grab to take over broadband is a corruption of net neutrality. It disrupted a market-based policy that worked fabulously smoothly for two decades with bipartisan approval.  \n",
       "1805356  I am writing in support of Net Neutrality rules. I make my living as a web developer. I remember specifically a time back in the age of dial-up, when I was working for a small privately-owned Internet Service Provider. When Verizon decided to start offering their own dial-up access product,  the cost to maintain dial-up access points (i.e. virtual phone numbers people could use to dial in to access our internet service) rose dramatically. Perhaps it was coincidence, bue the bottom line is th...  \n",
       "2887808                                                           Dear Commissioners:  I want to give my opinion on net neutrality regulations. I would like to request the commission to repeal Obama's scheme to take over the Internet. Internet users, rather than the FCC, ought to purchase the products we want. Obama's scheme to take over the Internet is a perversion of net neutrality. It reversed a pro-consumer framework that functioned supremely smoothly for a long time with broad bipartisan consensus.  \n",
       "1997919  To the Federal Communications Commission:  I have concerns about the FCC regulations on the Internet. I advocate Ajit Pai to reverse President Obama's power grab to take over Internet access. People like me, as opposed to Washington bureaucrats, should use whichever services they desire. President Obama's power grab to take over Internet access is a exploitation  of the open Internet. It disrupted a light-touch framework that functioned very well for many years with Republican and Democrat b...  \n",
       "1782180                                                                                                                                                I am an American citizen and voter. I appreciate that the FCC is inviting public comments on chairman Ajit Pai's intent to switch the legal regulation of the FCC from Title 2 to Title 1. I am writing to express my strong support for net neutrality , and I oppose the proposed switch to Title 1. \\n\\nThank you for considering my comments.\\nMiranda Glasbergen  \n",
       "1850861                                                                     Mr Pai:  With respect to the FCC's so-called Open Internet order. I recommend you to overturn Barack Obama's scheme to regulate Internet access. Individuals, as opposed to big government, should be able to purchase whatever products we desire. Barack Obama's scheme to regulate Internet access is a distortion of net neutrality. It reversed a hands-off approach that worked supremely well for a long time with both parties' backing.  \n",
       "1982121                            FCC commissioners,  I have concerns about NET NEUTRALITY. I strongly advocate the Federal Communications Commission to overturn The Obama/Wheeler power grab to control broadband. People like me, as opposed to Washington, should be empowered to buy the applications they prefer. The Obama/Wheeler power grab to control broadband is a perversion of net neutrality. It reversed a pro-consumer policy that performed remarkably smoothly for a long time with bipartisan approval.  \n",
       "587944                                                                       FCC commissioners,  I'm concerned about NET NEUTRALITY. I would like to implore Chairman Pai to reverse Barack Obama's decision to regulate broadband. Individual citizens, as opposed to Washington, ought to enjoy whichever products we choose. Barack Obama's decision to regulate broadband is a betrayal of the open Internet. It disrupted a free-market policy that functioned very smoothly for many years with both parties' support.  \n",
       "2260691                                                                                                                                                                                                                            I urge the FCC to continue upholding Obama era net neutrality regulations. I think these regulations are important to allowing government oversight to internet service providers, which otherwise might profit at the expense of access to the internet and international communication.  \n",
       "2603882                                    Mr Pai:  I would like to comment on the FCC regulations on the Internet. I want to advocate you to undo The previous administration's scheme to take over broadband. Internet users, as opposed to big government, should be able to use which products they choose. The previous administration's scheme to take over broadband is a perversion of net neutrality. It broke a hands-off approach that performed very, very smoothly for many years with both parties' consensus.  \n",
       "2185523                                                                                                                                                                                                                                                                                                                                                                                                                                                          The current Title II classification should be kept in place  \n",
       "742775                                                                                                                                                                                                                                                                                                                                                                                                                    Keep the open internet, they are trying to cash money out of blocking or making thing a lot slower  \n",
       "2582610                                                          To the Federal Communications Commission:  I'm a voter worried about internet regulations. I strongly demand you to rescind Barack Obama's order to take over the Internet. Americans, rather than Washington, should buy whichever products they choose. Barack Obama's order to take over the Internet is a betrayal of the open Internet. It undid a hands-off policy that performed very, very well for decades with Republican and Democrat consensus.  \n",
       "2289997                                                                                                                                                                                                                                                                                                                            I strongly believe in the Title II classification required for oversight where Net Neutrality protections are concerned. This is necessary for a strong, free internet for every citizen.  \n",
       "1559919  The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked FaceTime, T...  \n",
       "2579707                                                                               I'm a voter worried about the FCC regulations on the Internet. I ask Ajit Pai to rescind Tom Wheeler's policy to control broadband. Individual citizens, rather than Washington, deserve to use the applications we want. Tom Wheeler's policy to control broadband is a corruption of net neutrality. It disrupted a market-based framework that functioned exceptionally successfully for two decades with nearly universal backing.  \n",
       "2818845  To the Federal Communications Commission:  With respect to the FCC's Open Internet order. I strongly demand the government to overturn President Obama's power grab to control Internet access. People like me, rather than the FCC Enforcement Bureau, should be free to buy the applications they desire. President Obama's power grab to control Internet access is a betrayal of the open Internet. It reversed a market-based system that worked very, very successfully for decades with bipartisan consen...  \n",
       "2566708                                         Dear Commissioners:  I want to give my opinion on Internet regulation and net neutrality. I want to urge you to repeal The Obama/Wheeler policy to regulate Internet access. Americans, not unelected bureaucrats, should be able to select whichever services they want. The Obama/Wheeler policy to regulate Internet access is a betrayal of net neutrality. It reversed a hands-off approach that worked very, very smoothly for a long time with both parties' support.  \n",
       "1634760                                                                                                                                                                                                                                                                         The government would best serve its citizens by repealing/removing regulations and subsidies.  Small government and few regulations is better and cheaper and allows for more innovation.\\n\\nObama's Internet takeover should be repealed.\\n  \n",
       "1848023  As an Internet user, I'm asking the FCC to protect the net neutrality protections currently in place. \\n\\nThe FCC should throw out Chairman Ajit Pai's proposal to give the ISP monopolies like Comcast, AT&T, and Verizon the authority to create Internet fast lanes, stripping Internet users of the meaningful access and privacy protections we fought for and just recently won. \\n\\nI'm concerned about ISPs being allowed to discriminate against certain types of data or websites, because users will ...  \n",
       "1428928                                                                                                                           PLEASE HELP SUSTAIN WHATEVER INTEGRITY IS LEFT IN THIS GREAT NATION.  THE INTERNET IS OUR LAST BASTION OF FREEDOM.  Do not let the lobbyists and corporate greed siphon the remaining dollars from our already gutted middle and lower class families through package upgrade fee implementations and other bandwidth throttling practices.  This is sheer madness.  It must be stopped.    \n",
       "2393200                       To the FCC:  I'm concerned about Internet regulation and net neutrality. I strongly urge the Federal Communications Commission to reverse The Obama/Wheeler decision to take over broadband. Individuals, rather than Washington bureaucrats, should select which services we desire. The Obama/Wheeler decision to take over broadband is a betrayal of the open Internet. It reversed a market-based framework that performed supremely successfully for decades with both parties' backing.  \n",
       "2716380                                              Chairman Pai:  I am concerned about restoring Internet freedom. I encourage the Federal Communications Commission to rescind Barack Obama's power grab to control the Internet. Americans, not unelected bureaucrats, deserve to buy whatever products we desire. Barack Obama's power grab to control the Internet is a distortion of net neutrality. It ended a light-touch approach that functioned very successfully for many years with nearly universal approval.  \n",
       "2400235                                                                       Dear FCC,  I have thoughts on Title 2 and net neutrality. I want to demand you to undo The Obama/Wheeler policy to regulate Internet access. Americans, not big government, deserve to use the applications they prefer. The Obama/Wheeler policy to regulate Internet access is a exploitation  of the open Internet. It reversed a hands-off approach that functioned very, very well for a long time with Republican and Democrat approval.  \n",
       "2101661                                                                                                                                                                                                                                                                                                                                                                                                                                             I STRONGLY support \"Net Neutrality\" backed by Title II oversight of ISPs  \n",
       "1406289                                                                                                                                                                                                                                                                                                                                                           Protect Net Neutrality, allow me, my family and my global neighbors to choose what we read and where we go on the net, instead of profit margins choosing.  \n",
       "2747741                                     To the FCC:  I'd like to share my thoughts on net neutrality and Title II. I want to recommend Chairman Pai to rescind The previous administration's policy to control the web. Americans, rather than the FCC, should purchase the applications we desire. The previous administration's policy to control the web is a perversion of the open Internet. It stopped a light-touch framework that worked exceptionally successfully for two decades with both parties' approval.  \n",
       "2308441                                                                                                                                                                                                                                                                                                                                                                  Please save net neutrality. Protect our right to communication and we will be fighting for something that will have a lasting affect for centuries.  \n",
       "1320821  The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked FaceTime, T...  \n",
       "2159377                                                                                                                                                                                                                                                                                                                                          I specifically support strong net neutrality backed by TITLE II oversight of ISP's. \\nKeeping Title 2 is the only way to ensure a fair and open internet for all Americans.  \n",
       "428094                                                Dear Mr. Pai,  I'm a voter worried about net neutrality and Title II. I'd like to ask Ajit Pai to undo The previous administration's policy to regulate Internet access. Individual citizens, not Washington, should purchase which services we want. The previous administration's policy to regulate Internet access is a exploitation  of net neutrality. It undid a pro-consumer policy that worked remarkably well for a long time with nearly universal support.  \n",
       "2348222                                                                          Dear Commissioners:  Hi, I'd like to comment on an open Internet. I want to request you to rescind The Obama/Wheeler order to control Internet access. Citizens, rather than the FCC, ought to buy which services they choose. The Obama/Wheeler order to control Internet access is a perversion of net neutrality. It broke a light-touch system that worked exceptionally smoothly for two decades with Republican and Democrat backing.  \n",
       "1176716  The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked FaceTime, T...  \n",
       "1709464                                                                          Dear Express Restoring Internet Freedom,\\n\\nI support Chairman Ajit PaiÕs rollback of ObamaÕs internet takeover. \\n\\nTitle II regulation gives too much power to the government over the future of the internet. \\n\\nIt has harmed innovation and investment. It also deters competition. \\n\\nMarkets and consumers should regulate the InternetÕs future, not government.\\n\\nRegards,\\nJanet Barrett\\n1289 Corbett Ln\\nMelbourne, FL 32904  \n",
       "537410                                                                                                                                                                                                                                                                                                                                                                                                                         I want you to preserve Net Neutrality and maintain Title II for the benefit of all Americans.  \n",
       "405464      Dear Chairman Pai,  I would like to comment on Internet Freedom. I would like to demand the government to repeal The previous administration's scheme to take over Internet access. Americans, as opposed to so-called experts, should be able to enjoy whatever services they choose. The previous administration's scheme to take over Internet access is a distortion of the open Internet. It broke a free-market approach that functioned exceptionally well for two decades with nearly universal backing.  \n",
       "1709184                                                                  Dear Express Restoring Internet Freedom,\\n\\nI support Chairman Ajit PaiÕs rollback of ObamaÕs internet takeover. \\n\\nTitle II regulation gives too much power to the government over the future of the internet. \\n\\nIt has harmed innovation and investment. It also deters competition. \\n\\nMarkets and consumers should regulate the InternetÕs future, not government.\\n\\nRegards,\\nRose Oxley\\n151 State Route 33 Ste 252\\nManalapan, NJ 07726  \n",
       "656844   The internet is one of the true frontiers of the 21st century. It has become a landscape of diversity that is fundamental to modern society. One of the pillars that contributes to this diversity is neutrality of its operating basis. If Internet neutrality is compromised then it will the resources available will become increasingly top heavy with the most wealthy and powerful organizations. The availability to the smaller producer of content is what makes for such a diverse and enjoyable inte...  \n",
       "974566   Dear FCC,\\n\\nNet neutrality is essential to maintain the free market spirit of our economy. Removing net neutrality will have a devastating effect on the ability of startups to prosper and will stifle competition and innovation. Most areas of the country only have one available ISP, and therefore, they have near monopoly power and it is the responsibility of the government in this case to intervene because the barriers to entry in the market are too high. Please reconsider your rule changes ...  \n",
       "530297   The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked FaceTime, T...  \n",
       "317557                                                       FCC commissioners,  I'm a voter worried about Internet Freedom. I want to ask Ajit Pai to repeal Barack Obama's power grab to take over the Internet. People like me, rather than the FCC, ought to enjoy whatever applications they choose. Barack Obama's power grab to take over the Internet is a exploitation  of the open Internet. It ended a free-market system that performed very, very smoothly for many years with Republican and Democrat backing.  \n",
       "215980                                                                                 To the FCC:  Regarding network neutrality regulations. I would like to ask the commission to overturn Obama's policy to regulate Internet access. Individual citizens, rather than big government, should select whatever products they choose. Obama's policy to regulate Internet access is a corruption of the open Internet. It ended a hands-off policy that functioned fabulously well for two decades with bipartisan backing.  \n",
       "2748401                             FCC commissioners,  Hi, I'd like to comment on the FCC's so-called Open Internet order. I would like to ask the government to rescind Barack Obama's plan to regulate Internet access. Internet users, rather than the FCC Enforcement Bureau, should use whatever products they prefer. Barack Obama's plan to regulate Internet access is a distortion of the open Internet. It ended a pro-consumer approach that worked exceptionally well for decades with both parties' consensus.  \n",
       "2411230                                                            Dear FCC,  I'm contacting you about the FCC's so-called Open Internet order. I want to suggest the commissioners to reverse Tom Wheeler's plan to control broadband. Internet users, as opposed to Washington, ought to buy which products we desire. Tom Wheeler's plan to control broadband is a distortion of net neutrality. It undid a market-based approach that functioned remarkably successfully for two decades with nearly universal approval.  \n",
       "241488   To the FCC:  I'm contacting you about the FCC regulations on the Internet. I would like to encourage the commission to repeal The previous administration's decision to control broadband. Individual citizens, as opposed to so-called experts, deserve to buy the applications we prefer. The previous administration's decision to control broadband is a distortion of the open Internet. It reversed a light-touch framework that functioned exceptionally smoothly for decades with nearly universal conse...  \n",
       "2388164                                                                    Dear Chairman Pai,  I'd like to share my thoughts on Net neutrality. I'd like to recommend Chairman Pai to reverse Obama's policy to control the web. Citizens, not the FCC Enforcement Bureau, deserve to use which products they want. Obama's policy to control the web is a exploitation  of the open Internet. It reversed a free-market policy that worked exceptionally successfully for a long time with Republican and Democrat backing.  \n",
       "497066                                                                                 FCC:  With respect to restoring Internet freedom. I want to ask the commission to overturn President Obama's policy to take over broadband. Individuals, as opposed to Washington, ought to enjoy whichever applications we prefer. President Obama's policy to take over broadband is a betrayal of the open Internet. It undid a light-touch system that functioned exceptionally smoothly for a long time with bipartisan support.  \n",
       "1219678  The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked FaceTime, T...  \n",
       "1059070                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               This is BAD! Very Bad!  \n",
       "689419   The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked FaceTime, T...  \n",
       "303003                                                           Chairman Pai:  I'm concerned about an open Internet. I'd like to suggest you to overturn The previous administration's plan to control the Internet. People like me, rather than unelected bureaucrats, should buy whatever applications they desire. The previous administration's plan to control the Internet is a distortion of net neutrality. It reversed a light-touch policy that functioned very well for a long time with both parties' approval.  \n",
       "1599015  The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\r\\n\\r\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\r\\n\\r\\nNow is not the time to let giant ISPs censor what we see and do online.\\r\\n\\r\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked...  \n",
       "1460903                                                                                                                                                                                                                                                                                                                                                                                            For integrity, truth, communication, education, growth personally and socially, for FREEDOM we NEED the OPEN INTERNET!!!!  \n",
       "786326   I’m calling on the FCC to stand up for net neutrality and safeguard Title II protections. \\r\\n\\r\\nThe FCC should reject Chairman Ajit Pai’s proposal to hand the government-subsidized telecom giants like Comcast, Verizon, and AT&T the legal cover to engage in data discrimination, stripping consumers of the meaningful privacy and access protections we worked for and won just two years ago. \\r\\n\\r\\nI’m concerned about ISPs being allowed to discriminate against certain types of data or websites ...  \n",
       "1584315  The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\r\\n\\r\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\r\\n\\r\\nNow is not the time to let giant ISPs censor what we see and do online.\\r\\n\\r\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked...  \n",
       "2026496                                                                              I’m worried that the protections that are in place will be weakened if changes are made to the way they’re enforced. I would support a new regulation style if it guarantees the same or better protections, but not if we lose any. I want all of the sites that I access on the web to be treated equally. Additionally I want my ISP to be fairly regulated so that no site or app can be blocked or pay for preferential treatment.  \n",
       "2751223                                 Dear Mr. Pai,  I'm very worried about Net neutrality. I'd like to demand the commissioners to undo The previous administration's power grab to take over Internet access. Citizens, as opposed to Washington, should enjoy the services they want. The previous administration's power grab to take over Internet access is a betrayal of the open Internet. It disrupted a light-touch framework that worked very, very well for a long time with Republican and Democrat approval.  \n",
       "1128901  The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\r\\n\\r\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\r\\n\\r\\nNow is not the time to let giant ISPs censor what we see and do online.\\r\\n\\r\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked...  \n",
       "2596386                                                              I'm contacting you about net neutrality and Title II. I strongly implore the Federal Communications Commission to undo Barack Obama's policy to regulate the Internet. People like me, rather than the FCC, should purchase the applications we choose. Barack Obama's policy to regulate the Internet is a betrayal of the open Internet. It stopped a light-touch framework that performed supremely well for decades with nearly universal approval.  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# eyeball above against a random sample\n",
    "data_by_dupe.loc[:,['dupe_count','text_data']].sample(60)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABKEAAAHeCAYAAABDixiUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X20pVV9J/jvD0pokICA1WiAtuiIsdWevFgBp5NM7GCA\nhG5hMmjTTkfMIjI9GHXS6W7KSc/Q0dApe2Xp6EpkmhYUSSISOt2Q0AQrqEm/8VKiEREdKgIKjVqh\nEHvy4gSy54+zKx4O91bdl9p1z636fNZ61n3OPr9n333Ovufe4st+nqdaawEAAACAkQ5Z6wEAAAAA\ncOATQgEAAAAwnBAKAAAAgOGEUAAAAAAMJ4QCAAAAYDghFAAAAADDCaEAAAAAGE4IBQAAAMBwQigA\nAAAAhtuw1gPYn5773Oe2TZs2rfUwAAAAAA4Yn/zkJ/+otbZxb3UHVQi1adOmbN++fa2HAQAAAHDA\nqKqHllLndDwAAAAAhhNCAQAAADCcEAoAAACA4YRQAAAAAAwnhAIAAABgOCEUAAAAAMMJoQAAAAAY\nTggFAAAAwHBCKAAAAACGE0IBAAAAMJwQCgAAAIDhhFAAAAAADCeEAgAAAGA4IRQAAAAAwwmhAAAA\nABhOCAUAAADAcEIoAAAAAIYTQgEAAAAwnBAKAAAAgOE2rPUA1sqmLTc/o+3BreeswUgAAAAADnxW\nQgEAAAAwnBAKAAAAgOGEUAAAAAAMJ4QCAAAAYDghFAAAAADDCaEAAAAAGE4IBQAAAMBwQigAAAAA\nhhNCAQAAADCcEAoAAACA4YRQAAAAAAwnhAIAAABgOCEUAAAAAMMJoQAAAAAYTggFAAAAwHBCKAAA\nAACGE0IBAAAAMJwQCgAAAIDhhFAAAAAADCeEAgAAAGA4IRQAAAAAwwmhAAAAABhOCAUAAADAcEIo\nAAAAAIYTQgEAAAAwnBAKAAAAgOGEUAAAAAAMJ4QCAAAAYDghFAAAAADDCaEAAAAAGE4IBQAAAMBw\nQigAAAAAhttrCFVVV1fV16rqs1Ntx1XVtqq6v389duq5t1XVjqr6QlWdNdX+8qq6pz/33qqq3n54\nVX2kt99RVZumjrmwf4/7q+rCqfZTeu2Ofuxhq38rAAAAABhlKSuhPpjk7Jm2LUlua62dmuS2/jhV\n9ZIkFyR5aT/mfVV1aD/miiRvTHJq33b3eVGSx1trL0zy7iTv7H0dl+SyJKcnOS3JZVNh1zuTvLsf\n83jvAwAAAIA5tdcQqrX2+0l2zTSfm+Savn9NkvOm2q9rrX2ztfZAkh1JTquq5yc5urV2e2utJfnQ\nzDG7+7ohyRl9ldRZSba11na11h5Psi3J2f25H+61s98fAAAAgDm00mtCndBae7TvfyXJCX3/xCRf\nnqp7uLed2Pdn2592TGvtySRPJDl+D30dn+TrvXa2r2eoqourantVbd+5c+dyXiMAAAAA+8iqL0ze\nVza1fTCWIVprV7bWNrfWNm/cuHGthwMAAABwUFppCPXVfopd+tev9fZHkpw8VXdSb3uk78+2P+2Y\nqtqQ5Jgkj+2hr8eSPKfXzvYFAAAAwBxaaQh1U5Ldd6u7MMmNU+0X9DvenZLJBcjv7KfufaOqXtGv\n6fT6mWN293V+ko/11VW3Jjmzqo7tFyQ/M8mt/bmP99rZ7w8AAADAHNqwt4Kq+nCSVyZ5blU9nMkd\n67Ymub6qLkryUJLXJklr7d6quj7J55I8meRNrbWneleXZHKnvSOS3NK3JLkqybVVtSOTC6Bf0Pva\nVVXvSHJXr3t7a233BdIvTXJdVf1Ckk/1PgAAAACYUzVZWHRw2Lx5c9u+fXuSZNOWm5/x/INbz9nf\nQwIAAABY16rqk621zXurW/WFyQEAAABgb4RQAAAAAAwnhAIAAABgOCEUAAAAAMMJoQAAAAAYTggF\nAAAAwHBCKAAAAACGE0IBAAAAMJwQCgAAAIDhhFAAAAAADCeEAgAAAGA4IRQAAAAAwwmhAAAAABhO\nCAUAAADAcEIoAAAAAIYTQgEAAAAwnBAKAAAAgOGEUAAAAAAMJ4QCAAAAYDghFAAAAADDCaEAAAAA\nGE4IBQAAAMBwQigAAAAAhhNCAQAAADCcEAoAAACA4YRQAAAAAAwnhAIAAABgOCEUAAAAAMMJoQAA\nAAAYTggFAAAAwHBCKAAAAACGE0IBAAAAMJwQCgAAAIDhhFAAAAAADCeEAgAAAGA4IRQAAAAAwwmh\nAAAAABhOCAUAAADAcEIoAAAAAIYTQgEAAAAwnBAKAAAAgOGEUAAAAAAMJ4QCAAAAYDghFAAAAADD\nCaEAAAAAGE4IBQAAAMBwQigAAAAAhhNCAQAAADCcEAoAAACA4YRQAAAAAAwnhAIAAABgOCEUAAAA\nAMMJoQAAAAAYTggFAAAAwHBCKAAAAACGE0IBAAAAMJwQCgAAAIDhhFAAAAAADCeEAgAAAGA4IRQA\nAAAAwwmhAAAAABhOCAUAAADAcEIoAAAAAIbbsNYDWA82bbn5GW0Pbj1nDUYCAAAAsD5ZCQUAAADA\ncEIoAAAAAIZbVQhVVT9TVfdW1Wer6sNV9Veq6riq2lZV9/evx07Vv62qdlTVF6rqrKn2l1fVPf25\n91ZV9fbDq+ojvf2Oqto0dcyF/XvcX1UXruZ1AAAAADDWikOoqjoxyVuSbG6tvSzJoUkuSLIlyW2t\ntVOT3NYfp6pe0p9/aZKzk7yvqg7t3V2R5I1JTu3b2b39oiSPt9ZemOTdSd7Z+zouyWVJTk9yWpLL\npsMuAAAAAObLak/H25DkiKrakOTIJP81yblJrunPX5PkvL5/bpLrWmvfbK09kGRHktOq6vlJjm6t\n3d5aa0k+NHPM7r5uSHJGXyV1VpJtrbVdrbXHk2zLt4IrAAAAAObMikOo1tojSX4pyZeSPJrkidba\nR5Oc0Fp7tJd9JckJff/EJF+e6uLh3nZi359tf9oxrbUnkzyR5Pg99AUAAADAHFrN6XjHZrJS6ZQk\n357k2VX1D6Zr+sqmtqoRrlJVXVxV26tq+86dO9dyKAAAAAAHrdWcjveqJA+01na21v48yW8m+VtJ\nvtpPsUv/+rVe/0iSk6eOP6m3PdL3Z9ufdkw/5e+YJI/toa9naK1d2Vrb3FrbvHHjxhW+VAAAAABW\nYzUh1JeSvKKqjuzXaTojyX1Jbkqy+251Fya5se/flOSCfse7UzK5APmd/dS9b1TVK3o/r585Zndf\n5yf5WF9ddWuSM6vq2L4i68zeBgAAAMAc2rDSA1trd1TVDUnuTvJkkk8luTLJUUmur6qLkjyU5LW9\n/t6quj7J53r9m1prT/XuLknywSRHJLmlb0lyVZJrq2pHkl2Z3F0vrbVdVfWOJHf1ure31nat9LUA\nAAAAMNaKQ6gkaa1dluSymeZvZrIqaqH6y5NcvkD79iQvW6D9z5K8ZpG+rk5y9TKHDAAAAMAaWM3p\neAAAAACwJEIoAAAAAIYTQgEAAAAwnBAKAAAAgOGEUAAAAAAMJ4QCAAAAYDghFAAAAADDCaEAAAAA\nGE4IBQAAAMBwQigAAAAAhhNCAQAAADCcEAoAAACA4YRQAAAAAAwnhAIAAABgOCEUAAAAAMMJoQAA\nAAAYTggFAAAAwHBCKAAAAACGE0IBAAAAMJwQCgAAAIDhhFAAAAAADCeEAgAAAGA4IRQAAAAAwwmh\nAAAAABhOCAUAAADAcEIoAAAAAIYTQgEAAAAwnBAKAAAAgOGEUAAAAAAMJ4QCAAAAYDghFAAAAADD\nCaEAAAAAGE4IBQAAAMBwQigAAAAAhhNCAQAAADCcEAoAAACA4YRQAAAAAAwnhAIAAABgOCEUAAAA\nAMMJoQAAAAAYTggFAAAAwHBCKAAAAACGE0IBAAAAMJwQCgAAAIDhhFAAAAAADCeEAgAAAGA4IRQA\nAAAAwwmhAAAAABhOCAUAAADAcEIoAAAAAIYTQgEAAAAwnBAKAAAAgOGEUAAAAAAMJ4QCAAAAYDgh\nFAAAAADDCaEAAAAAGE4IBQAAAMBwQigAAAAAhhNCAQAAADCcEAoAAACA4YRQAAAAAAwnhAIAAABg\nOCEUAAAAAMMJoQAAAAAYTggFAAAAwHBCKAAAAACGE0IBAAAAMNyqQqiqek5V3VBVn6+q+6rqv6+q\n46pqW1Xd378eO1X/tqraUVVfqKqzptpfXlX39OfeW1XV2w+vqo/09juqatPUMRf273F/VV24mtcB\nAAAAwFirXQn1niS/01p7cZLvSnJfki1JbmutnZrktv44VfWSJBckeWmSs5O8r6oO7f1ckeSNSU7t\n29m9/aIkj7fWXpjk3Une2fs6LsllSU5PclqSy6bDLgAAAADmy4pDqKo6Jsn/kOSqJGmt/X+tta8n\nOTfJNb3smiTn9f1zk1zXWvtma+2BJDuSnFZVz09ydGvt9tZaS/KhmWN293VDkjP6Kqmzkmxrre1q\nrT2eZFu+FVwBAAAAMGdWsxLqlCQ7k3ygqj5VVe+vqmcnOaG19miv+UqSE/r+iUm+PHX8w73txL4/\n2/60Y1prTyZ5Isnxe+jrGarq4qraXlXbd+7cuaIXCgAAAMDqrCaE2pDke5Nc0Vr7niR/nH7q3W59\nZVNbxfdYtdbala21za21zRs3blzLoQAAAAActFYTQj2c5OHW2h398Q2ZhFJf7afYpX/9Wn/+kSQn\nTx1/Um97pO/Ptj/tmKrakOSYJI/toS8AAAAA5tCKQ6jW2leSfLmqvrM3nZHkc0luSrL7bnUXJrmx\n79+U5IJ+x7tTMrkA+Z391L1vVNUr+vWeXj9zzO6+zk/ysb666tYkZ1bVsf2C5Gf2NgAAAADm0IZV\nHv/mJL9WVYcl+WKSn8wk2Lq+qi5K8lCS1yZJa+3eqro+k6DqySRvaq091fu5JMkHkxyR5Ja+JZOL\nnl9bVTuS7Mrk7nppre2qqnckuavXvb21tmuVrwUAAACAQVYVQrXWPp1k8wJPnbFI/eVJLl+gfXuS\nly3Q/mdJXrNIX1cnuXo54wUAAABgbazmmlAAAAAAsCRCKAAAAACGE0IBAAAAMJwQCgAAAIDhhFAA\nAAAADCeEAgAAAGA4IRQAAAAAwwmhAAAAABhOCAUAAADAcEIoAAAAAIYTQgEAAAAwnBAKAAAAgOGE\nUAAAAAAMt2GtB3Cg2bTl5me0Pbj1nDUYCQAAAMD8sBIKAAAAgOGshFpDVk0BAAAABwsroQAAAAAY\nTggFAAAAwHBCKAAAAACGE0IBAAAAMJwQCgAAAIDhhFAAAAAADCeEAgAAAGA4IRQAAAAAwwmhAAAA\nABhOCAUAAADAcEIoAAAAAIYTQgEAAAAwnBAKAAAAgOGEUAAAAAAMJ4QCAAAAYDghFAAAAADDCaEA\nAAAAGE4IBQAAAMBwQigAAAAAhhNCAQAAADCcEAoAAACA4YRQAAAAAAwnhAIAAABgOCEUAAAAAMMJ\noQAAAAAYTggFAAAAwHBCKAAAAACGE0IBAAAAMNyGtR4AS7Npy83PaHtw6zlrMBIAAACA5bMSCgAA\nAIDhhFAAAAAADCeEAgAAAGA4IRQAAAAAwwmhAAAAABjO3fEOQO6kBwAAAMwbK6EAAAAAGE4IBQAA\nAMBwQigAAAAAhhNCAQAAADCcEAoAAACA4YRQAAAAAAwnhAIAAABgOCEUAAAAAMMJoQAAAAAYTggF\nAAAAwHAb1noArK1NW25+RtuDW89Zg5EAAAAABzIroQAAAAAYTggFAAAAwHBCKAAAAACGE0IBAAAA\nMJwQCgAAAIDhVh1CVdWhVfWpqvrt/vi4qtpWVff3r8dO1b6tqnZU1Req6qyp9pdX1T39ufdWVfX2\nw6vqI739jqraNHXMhf173F9VF672dQAAAAAwzr5YCfXWJPdNPd6S5LbW2qlJbuuPU1UvSXJBkpcm\nOTvJ+6rq0H7MFUnemOTUvp3d2y9K8nhr7YVJ3p3knb2v45JcluT0JKcluWw67AIAAABgvqwqhKqq\nk5Kck+T9U83nJrmm71+T5Lyp9utaa99srT2QZEeS06rq+UmObq3d3lprST40c8zuvm5IckZfJXVW\nkm2ttV2ttceTbMu3gisAAAAA5sxqV0L9X0n+aZK/mGo7obX2aN//SpIT+v6JSb48Vfdwbzux78+2\nP+2Y1tqTSZ5Icvwe+nqGqrq4qrZX1fadO3cu68UBAAAAsG+sOISqqr+T5GuttU8uVtNXNrWVfo99\nobV2ZWttc2tt88aNG9dyKAAAAAAHrdWshPr+JK+uqgeTXJfkh6vqV5N8tZ9il/71a73+kSQnTx1/\nUm97pO/Ptj/tmKrakOSYJI/toS8AAAAA5tCKQ6jW2ttaaye11jZlcsHxj7XW/kGSm5LsvlvdhUlu\n7Ps3Jbmg3/HulEwuQH5nP3XvG1X1in69p9fPHLO7r/P792hJbk1yZlUd2y9IfmZvAwAAAGAObRjQ\n59Yk11fVRUkeSvLaJGmt3VtV1yf5XJInk7yptfZUP+aSJB9MckSSW/qWJFclubaqdiTZlUnYldba\nrqp6R5K7et3bW2u7BrwWAAAAAPaBfRJCtdY+keQTff+xJGcsUnd5kssXaN+e5GULtP9Zktcs0tfV\nSa5e6ZgBAAAA2H9We3c8AAAAANirEafjcYDatOXmZ7Q9uPWcNRgJAAAAsN5YCQUAAADAcEIoAAAA\nAIYTQgEAAAAwnGtCMcRC149KXEMKAAAADlZWQgEAAAAwnBAKAAAAgOGEUAAAAAAMJ4QCAAAAYDgh\nFAAAAADDCaEAAAAAGE4IBQAAAMBwQigAAAAAhhNCAQAAADCcEAoAAACA4YRQAAAAAAwnhAIAAABg\nuA1rPQDYtOXmBdsf3HrOfh4JAAAAMIqVUAAAAAAMJ4QCAAAAYDghFAAAAADDCaEAAAAAGE4IBQAA\nAMBwQigAAAAAhhNCAQAAADCcEAoAAACA4YRQAAAAAAwnhAIAAABguA1rPQBYjk1bbl6w/cGt5+zn\nkQAAAADLYSUUAAAAAMMJoQAAAAAYTggFAAAAwHBCKAAAAACGE0IBAAAAMJwQCgAAAIDhhFAAAAAA\nDCeEAgAAAGA4IRQAAAAAwwmhAAAAABhOCAUAAADAcEIoAAAAAIYTQgEAAAAwnBAKAAAAgOGEUAAA\nAAAMJ4QCAAAAYDghFAAAAADDCaEAAAAAGE4IBQAAAMBwQigAAAAAhhNCAQAAADCcEAoAAACA4Tas\n9QBglE1bbl6w/cGt5+znkQAAAABWQgEAAAAwnBAKAAAAgOGEUAAAAAAMJ4QCAAAAYDgXJocs7yLm\nLngOAAAAy2clFAAAAADDCaEAAAAAGM7peDCQU/cAAABgwkooAAAAAIazEgrmhFVTAAAAHMiEULAO\nCawAAABYb5yOBwAAAMBwQigAAAAAhltxCFVVJ1fVx6vqc1V1b1W9tbcfV1Xbqur+/vXYqWPeVlU7\nquoLVXXWVPvLq+qe/tx7q6p6++FV9ZHefkdVbZo65sL+Pe6vqgtX+joAAAAAGG8114R6MsnPttbu\nrqpvS/LJqtqW5A1Jbmutba2qLUm2JLm0ql6S5IIkL03y7Ul+t6pe1Fp7KskVSd6Y5I4k/z7J2Ulu\nSXJRksdbay+sqguSvDPJ36uq45JclmRzkta/902ttcdX8XrggOT6UQAAAMyDFa+Eaq092lq7u+//\ntyT3JTkxyblJrull1yQ5r++fm+S61to3W2sPJNmR5LSqen6So1trt7fWWpIPzRyzu68bkpzRV0md\nlWRba21XD562ZRJcAQAAADCH9sk1ofppct+TyUqmE1prj/anvpLkhL5/YpIvTx32cG87se/Ptj/t\nmNbak0meSHL8HvpaaGwXV9X2qtq+c+fOFbw6AAAAAFZr1SFUVR2V5N8k+d9aa9+Yfq6vbGqr/R6r\n0Vq7srW2ubW2eePGjWs5FAAAAICD1qpCqKp6ViYB1K+11n6zN3+1n2KX/vVrvf2RJCdPHX5Sb3uk\n78+2P+2YqtqQ5Jgkj+2hLwAAAADm0GrujldJrkpyX2vtXVNP3ZRk993qLkxy41T7Bf2Od6ckOTXJ\nnf3UvW9U1St6n6+fOWZ3X+cn+VhfXXVrkjOr6th+970zexsAAAAAc2g1d8f7/iQ/keSeqvp0b/vf\nk2xNcn1VXZTkoSSvTZLW2r1VdX2Sz2VyZ7039TvjJcklST6Y5IhM7op3S2+/Ksm1VbUjya5M7q6X\n1tquqnpHkrt63dtba7tW8VoAAAAAGGjFIVRr7T8mqUWePmORYy5PcvkC7duTvGyB9j9L8ppF+ro6\nydVLHS8AAAAAa2ef3B0PAAAAAPZECAUAAADAcEIoAAAAAIYTQgEAAAAwnBAKAAAAgOGEUAAAAAAM\nt2GtBwDMj01bbl6w/cGt5+znkQAAAHCgsRIKAAAAgOGEUAAAAAAMJ4QCAAAAYDghFAAAAADDCaEA\nAAAAGE4IBQAAAMBwQigAAAAAhhNCAQAAADCcEAoAAACA4YRQAAAAAAwnhAIAAABguA1rPQBg/dq0\n5eZntD249Zw1GAkAAADzzkooAAAAAIYTQgEAAAAwnBAKAAAAgOGEUAAAAAAM58LkwH7hIuYAAAAH\nNyuhAAAAABhOCAUAAADAcEIoAAAAAIYTQgEAAAAwnAuTA3NnORcxn4daAAAA9k4IBbBK8xCErbZ2\nT/UAAAD7gtPxAAAAABhOCAUAAADAcE7HA2BRTt0DAAD2FSEUAPuEwAoAANgTp+MBAAAAMJyVUADs\nd1ZNAQDAwUcIBcBcE1gBAMCBwel4AAAAAAxnJRQABwyrpgAAYH5ZCQUAAADAcFZCAXBQWs6qKSus\nAABg9YRQALAPCawAAGBhTscDAAAAYDghFAAAAADDCaEAAAAAGM41oQBgjbh+FAAABxMhFACsEwuF\nVosFVsupBQCA/cHpeAAAAAAMZyUUABzkrJoCAGB/EEIBAEsmsAIAYKWcjgcAAADAcFZCAQBDWDUF\nAMA0IRQAsOYEVgAABz4hFACwrgisAADWJyEUAHDAWk5gJdwCABhLCAUAsEyrDbf2VA8AcKASQgEA\nzInlBFbCLQBgvTlkrQcAAAAAwIHPSigAgAPcqBVWVm4BAMthJRQAAAAAw1kJBQDAXLFqCgAOTEIo\nAADWLYEVAKwfQigAAA4KAisAWFtCKAAAWMBCodVigdVyagHgYCWEAgCA/UhgBcDBSggFAABzatRq\nLEEYAGtBCAUAACxKuAXAvrKuQ6iqOjvJe5IcmuT9rbWtazwkAABgCfbnKq/F6tdbLcB6t25DqKo6\nNMmvJPmRJA8nuauqbmqtfW5tRwYAALDvzUMQNg+1i9Wr3XMtzIN1G0IlOS3JjtbaF5Okqq5Lcm4S\nIRQAAABMsfpQ7UprF6tfSeBZrbVlHzQPqur8JGe31n6qP/6JJKe31n56pu7iJBf3h9+Z5AszXT03\nyR8t8dvOQ+28jEOt2oOldl7GoVbtwVI7L+NQq/ZgqZ2XcahVe7DUzss41Krd17UvaK1t3OvRrbV1\nuSU5P5PrQO1+/BNJfnkF/WxfT7XzMg61ag+W2nkZh1q1B0vtvIxDrdqDpXZexqFW7cFSOy/jUKt2\nf9QutB2S9euRJCdPPT6ptwEAAAAwZ9ZzCHVXklOr6pSqOizJBUluWuMxAQAAALCAdXth8tbak1X1\n00luTXJokqtba/euoKsr11ntvIxDrdqDpXZexqFW7cFSOy/jUKv2YKmdl3GoVXuw1M7LONSq3R+1\nz7BuL0wOAAAAwPqxnk/HAwAAAGCdEEIBAAAAMJwQCgAAAIDhDroQqqpeXFWXVtV7+3ZpVf2NfdTv\nGVV11Ez72QvUnlZV39f3X1JV/6iqfmyJ3+dDS6z7gd7vmQs8d3pVHd33j6iqn6+q36qqd1bVMQvU\nv6WqTl7C9zysql5fVa/qj19XVb9cVW+qqmctUP/Xq+ofV9V7qupdVfUPd48L5kVV/dVB/R4/ol++\nxdytT+Zt/TJ365N5W7/M3fpk3tYvc7dvHFQhVFVdmuS6JJXkzr5Vkg9X1ZZl9vWTU/tvSXJjkjcn\n+WxVnTtV+i9mjrssyXuTXFFVv5jkl5M8O8mWqvq5mdqbZrbfSvLjux/P1N45tf/G3u+3Jblsgdd2\ndZI/6fvvSXJMknf2tg8s8HLfkeSOqvoPVXVJVW1c5G35QJJzkry1qq5N8pokdyT5viTvnxnvW5L8\n30n+Sn/+8CQnJ7m9ql65SP/r2sH4S6uqjqmqrVX1+araVVWPVdV9ve05y+jnlpnHR1fVL1bVtVX1\nupnn3jfz+HlVdUVV/UpVHV9V/7yq7qmq66vq+TO1x81sxye5s6qOrarjZmrPnto/pqquqqrPVNWv\nV9UJM7Vbq+q5fX9zVX0xk8/UQ1X1QzO1d1fVP6uq71jC+7K5qj5eVb9aVSdX1baqeqKq7qqq71mg\n/qiqentV3dvrdlbV7VX1hgVqzV3mY+7M21/WHrDzNvWazN06mzvz9pfPr/m89Xpzl/U3d+btL2vN\n297Hd8sCbeYu4+ZuiNbaQbMl+X+SPGuB9sOS3L/Mvr40tX9PkqP6/qYk25O8tT/+1Mxx9yQ5NMmR\nSb6R5OjefkSSz8zU3p3kV5O8MskP9a+P9v0fmqn91NT+XUk29v1nJ7lnpva+6e8x89ynF3itn8ok\nsDwzyVVJdib5nSQXJvm2qbrP9K8bknw1yaH9cS3w2u6Zev7IJJ/o+39t9j3r7cck2Zrk80l2JXks\nyX297TnLmLdbZh4fneQXk1yb5HUzz71v5vHzklyR5FeSHJ/kn/fXcX2S58/UHjezHZ/kwSTHJjlu\npvbsmdd5VZLPJPn1JCfM1G5N8ty+vznJF5PsSPLQAj8Tdyf5Z0m+Ywnvy+YkH+8/bycn2Zbkif6z\n9D0ztUcleXuSe3vNziS3J3nDAv3emuTSJM+beR8vTfLRmdrvXWR7eZJHZ2r/TX8vzktyU398+CI/\n07+TSUC8pb+vl/bX+OYkN87U/kWSB2a2P+9fvzj7/k7tvz/JLyR5QZKfSfLvZn/ep/Y/nuT7+v6L\nkmyfqX0gyS8l+VImQfnPJPn2RebtziQ/muTvJ/lykvN7+xlJ/ssC9TcmeUOSk5L8oyT/R5JTk1yT\n5F+Yu/l50IR5AAAKE0lEQVScO/N24M+buVu/c2fe5mfezN36nTvzZt5WOm/mbr/9G6WSnJ7kx/t2\nepJaaBxL2VZ00HrdMgkwXrBA+wuSfGGB9s8sst2T5JtTdffOHHdU/wF/V2ZCnTw9LJoNqGZrD+k/\naNuSfHdv++Iir+0PMgk4jl/gwzb7fX4jyU/2/Q8k2Tz1g3/XAn3P9vesJK9O8uEkO6faP5tJoHds\nkv+WHrZkstrpvpk+7sm3fjkcO/2BS/LZBcbgD05bf7+0ssDnarHnkjyV5GP9dc1uf7qXz8rPJflP\n2cvPf6bC40X6+dk+z39z+n1cZPx376Gf2cf3JdnQ929fbE4X6PcHk7wvyVf6+3DxMl7bQmHuH8w8\nvqt/PSTJ583dfM6deTvw583crd+5M2/zM2/mbv3OnXkzbyudN3O3X+buzEwWPdySyX+rvr+/LzuS\nnLnYvO5pW/YB63lLcvbUG3hl33a/gWcvUP/VJN+dSSAwvW1K8l+n6j6WHhJNtW1I8qEkT82035Hk\nyN2TPNV+zOyHZOq5kzIJjn559gdqqubBTFbFPNC/Pr+3H7XAD/4xST6Y5A/7eP68H/N7Sb5rgb6f\n8Ud+6rkjp/Z/pvfzUJK3JLktyb/OJHC6bOa4t2YS5PzrTMLB3aHYxiS/v8D38QenrctfWh9N8k8z\ntaIryQmZBHi/O1P72SSnLvIefXmB9+GQmbY3ZLI666HFxpvkF/b0ns183t6VySmtiwW/D2cSwv1s\nJp+7mnpuduXfm/t78cOZrKB7TyYrGn8+ybWLzdtU26GZ/P76wEz7f8nkD8NrMvncndfbfygzoWRv\n/89JfqDvvzrJrXv4HM373H1mge91QM6deTvw583crd+5M28rmreFwiN/5w7iuTNvfleudN7249y9\n+wCeu3P3Mnf3Jdm0wDhOycxCk6Vuyz5gvW+Z/IfyK5L8T317RfppYQvUXrV7chZ47tdnfjift0jd\n9888PnyRuudmKrBYpOacLLA8bi/HHJnklEWeOzrJd2WyQuiEPfTxomV8v29PX22T5DlJzk9y2iK1\nL+3Pv3gJ/c77H5yDJszI8v7gHJvJ9cY+n+TxTE6lvK+3zZ6WeH6S71zkPTpv5vG/TPKqBerOzsyp\ntZmcOnjUArUvTHLDHn7mXp3JaYZfWeT5y2a23afAPi/Jhxaof2WSj2Ryeus9Sf59koszc4pwkuuW\n8Xn7rkxWCd6S5MX95+Hr/ef3by1Sf2efi/+4+/3OJPx9i7lb87l7vM/d7N+N2Xl70TLm7fE+b/9y\nHczbuft43v72AvP2v6xy3r57BfP29ezl8+Yzt18+cyuZu5V85szbGv2u7PX/nbnbZ78v99u/Uabm\nbffvywN13vb137lX7qd525e/K+/rc7ZP5m1OP3P/5zLm7u7su3+jjJq7+9MXQcy0H5Zkx1LH97Rj\nV3KQzba/tzz9D87sL65jZ2rn7Q/O/vpH3oaZutH/UNjrL63e/uIkr5p977Lw6sMXZ3IK4Gpqf3Rf\n9ZvJtdpeNni8q639G0utnapf6nyclm+d8vnSTILSH1uk3+nal2QSrK5l7d/M5Jpo+7rfffk+nL6M\nfk9far8LHHvtUup67TN+5+zP2v6Z+431Mt7B7+8P9p+JvS51T/ID/WdiLWt/sH/mRvQ74n3YJ/32\nz+Yxff/ITP6t8NuZ/PvkmAVqp68F+vYkv7WH2mOWUTvd788vsfbITP4d9LtLGMORyxjD6PdhwX4X\nGfOe3ou3JDl5iZ/HuavN1L9R1sN492HtYZlcl/ZV/fH/nMn1Wt+UZ/4H/OFJXj9V+7pMzizZn7WH\nzdT+RCZnbCzW74Ur7HdfvQ/LeX9naxfttz//HUn+SSY36Xp3kn+Y/rtggdq/nuQfZ/LfJO/aU+1y\n61dR+68yWRCwWO3u17fUfvf5e7HAGP7XJdS+d0/9JnlbJv/deWmf49f1/U8ledtSPrezW/WOYd2q\nqp9srX1gnmur6ohMLhD+2fUw3n1VW5O7IL4pk8DwuzO5YP+N/bm7W2vfu8LaNyf56QG1o8Ywst9L\nMgln91i73Pqa3MnzRzM5tXhbJuHKJ5L8SCar3y7fQ+3pmZz2uZa1yxnvqNpR491TvzflmX44k3/w\nprX26j3UVib/N3wta5cz3lG1+328vf7O1tppff+nMvmd8e8yWaX6W621rYvUvrHX/ts1rr1kwHhH\nvg9LHe9PZfI7ebF+783kUgZPVtWVSf44k+tLntHbf3wPtX+S5IY1rh013v3e7wr6fqL394eZ3Azm\nN1prf5QFzNR+uNfuXOPa6weMdx7eh1/P5H/uLlb7a5n8PTwikxvkPDuTz+cZmZxNcOECtUdm8j9X\nj0rymyuoTWvtDfuodqXjHdXv3mqX0u/uudhTv29J8neS/H6SH8skwPh6kv8xySWttU+spHZk3732\n72ZyyZp92e9bMznDacR4R73HL8lkYcWJvemRJDe11j6XlVgombLZ1tOWRa6TpXbta7P8O0eqHVS7\nwr6XeidPtfNTu6y7qs5B7ajxrqv3YfYzmL3f5Vbt/NQu+Y7DasfWrqDvJd39We3c1S7nbtxq56d2\nyXdGX07tyL7Vjts2BNaBqvrMYk9lcm0otXNYm8n1tv7fJGmtPVhVr0xyQ1W9oNer3X+1y61/srX2\nVJI/qao/bK19ox/3p1X1F2rntnZzJjd++Lkk/6S19umq+tPW2u/lmV4+B7Wjxrve3ockOaSqjs3k\nP8QObX0VQGvtj6vqSbVzWzu9wvkPqmpza217Vb0okxu/qN1/tcutb621v8jkWpsfrapn5Vt3Cv6l\nTC4zoHb+ag+pqsMyCYWPzOSGS7syOeXsWXk6tfNTm0zCqqf680clSWvtS32+V1M7su+Dvraqjsnk\nlLzzkvzVJC3J1zK5Y/rW1trXF+h7z9rglMtm2xdblninQrVzV7ucO0eqHVi7gr6XfCdPtfNTO/Xc\nXu+qqna+arO8u9yqnZ/aJd9xWO3Y2hX0vaS7P6udu9rl3I1b7fzUvjVLvDP6cmpH9q32L2tvzeQa\nUM+banteb/voYp/dPW3LPsBmW4stS7xTodq5q13OnSPVDqxdQd9LvpOn2vmpXaBmyXdVVTs/tTPH\nLXqXW7XzU5sl3nFY7fjapdZneXd/Vjsntb1+OXfjVjs/tcu5M/qSa0f2rbYlM3c/X+pze9pcmBwA\nAACAp6mqj2ZyF9VrWmtf7W0nJHlDkh9prb1quX0esk9HCAAAAMCB4O8lOT7J71XVrqralckdnY9L\n8pqVdGglFAAAAABLNnUjiOUdJ4QCAAAAYKmq6kuttb+23OM2jBgMAAAAAOtXVX1msaeSnLCSPoVQ\nAAAAAMw6IclZSR6faa8k/3klHQqhAAAAAJj120mOaq19evaJqvrESjp0TSgAAAAAhjtkrQcAAAAA\nwIFPCAUAAADAcEIoAAAAAIYTQgEAAAAw3P8PaourgHr88/8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fd405bf7b00>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# quick plot of top 100 comments\n",
    "plt.figure(1, figsize=(20, 8))\n",
    "data_by_dupe.loc[:100,'dupe_count'].plot.bar()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "### end detour ###"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# some identified strings: you can add your own to the list below but it gets to be diminishing returns\n",
    "# also be careful about adding strings that are too likely to appear naturally -- try to include some 'quirks' in the signature string\n",
    "# this step is fairly manual and could definitely be better quantified but we accomplish the goal of cutting down our dataset\n",
    "\n",
    "signature_strings_lists =\\\n",
    "[\n",
    "    [\"he FCC\\'s Open Internet Rules (net neutrality rules) are extremely important to me\",\"he FCC Open Internet Rules (net neutrality rules) are extremely important to me\"],\n",
    "    [\"As an Internet user, I\\'m asking the FCC to protect the net neutrality protections currently in place. \\n\\nThe FCC should throw out Chairman Ajit Pai\\'s proposal to give the ISP monopolies like Comcast, AT&T, and Verizon the authority to create Internet fast lanes, stripping Internet users of the meaningful access and privacy protections we fought for and just recently won.\"],\n",
    "    [\"I am in favor of strong net neutrality under Title II of the Telecommunications Act.\\n\\n\\nSincerely,\\n\"],\n",
    "    [\"Dear FCC Chairman Ajit Pai,\\n\\nI support the existing Net Neutrality rules, which classify internet service providers under the Title II provision of the Telecommunications Act. Please DO NOT roll back these regulations.\\n\\nSincerely,\"],\n",
    "    [\"Dear Express Restoring Internet Freedom,\"],\n",
    "    [\"As an Etsy seller, net neutrality is essential to the success of my business and my ability to care for myself and my family.\"],\n",
    "    [\"Net neutrality is shit because without it our ISPs will spy on us!\"], #don't really get what these folks are getting at in this one :-/\n",
    "    [\"To whom it may concern:\\nI would like my video comment at the following URL to be considered for the record.\\n\\nhttp://video.battleforthenet.com/player/#\"],\n",
    "    [\"Net neutrality is a vital principle in a democratic society. The biggest losers under Chairman Pai's proposal are independent news outlets, small businesses, start-up blogs, grassroots activist groups...and everyone who uses the internet. And we won't stay quiet while mega-corporations gain control over the information we consume every day.\"],\n",
    "    [\"I am in favor of strong net neutrality under Title II of the Telecommunications Act. Sincerely,\"],\n",
    "    ['The Obama-era FCC regulations known as \"Title II\" enable the federal government to exert an extraordinary and unnecessary amount of regulatory control over the internet. This bureaucratic overreach impedes innovation, stifles investment and continues to create economic uncertainty for one of the largest sectors of the U.S. economy.\\n \\n I support Chairman Pai\\'s proposal to roll back Title II and restore the sensible regulatory framework that enjoyed broad bipartisan consensus and enabled the internet to thrive for more than two decades.\\n \\n I strongly urge all of the FCC Commissioners to support the Chairman\\'s proposal to repeal the harmful Title II internet takeover.'],\n",
    "    [\"Cable and phone companies provide access to the internet. They're telecommunications carriers. They do not (and should not) have the right to censor or slow down my speech and my access to online content.\"],\n",
    "    [\"The FCC's job is to stand up for consumers, not big cable companies.\"]\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 31.9 s, sys: 0 ns, total: 31.9 s\n",
      "Wall time: 32 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "## flag by signature string one in the list is a positive match\n",
    "for clusterid, signature_strings_list in enumerate(signature_strings_lists):\n",
    "    masks = []\n",
    "    for signature_string in signature_strings_list:\n",
    "        masks.append(data['text_data'].str.contains(signature_string, regex=False))\n",
    "    ##flag the dataframe with signature string\n",
    "    ##first combine masks\n",
    "    mask = masks[0]\n",
    "    if len(masks) > 1:\n",
    "        for t_mask in masks[1:]:\n",
    "            mask = mask | t_mask\n",
    "    data.loc[mask,'level_0_cluster'] = clusterid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# after a lot of digging and iterations through the EDA I was able to pick out 2 mad-libs\n",
    "# these are the most egregious examples that appear on their face to be bot campaigns, by text alone\n",
    "regex_dicts = [\n",
    "    {\n",
    "        'a':r\"(My comments re:|I'm concerned about|Hi, I'd like to comment on|With respect to|I'm very worried about|Regarding|I have thoughts on|I want to give my opinion on|I'm a voter worried about|I am a voter worried about|In the matter of|I'm contacting you about|I would like to comment on|I want to give my opinion on|I'd like to share my thoughts on|I'm very concerned about|I am concerned about|I have concerns about|In reference to|With respect to) (the FCC's so-called Open Internet order\\.|Net Neutrality and Title II\\.|net neutrality and Title II\\.|the Open Internet order\\.|restoring Internet freedom\\.|an open Internet\\.|Network Neutrality\\.|the FCC's Open Internet order\\.|network neutrality regulations\\.|Internet regulation\\.|Internet regulation and net neutrality\\.|net neutrality regulations\\.|Internet freedom\\.|net neutrality rules\\.|Net neutrality\\.|internet regulations\\.|Title 2 and net neutrality\\.|net neutrality\\.|regulations on the Internet\\.|the Obama takeover of the Internet\\.|the FCC rules on the Internet\\.|the FCC regulations on the Internet\\.|Internet Freedom\\.|the future of the Internet\\.|Title II rules\\.|NET NEUTRALITY\\.) \",\n",
    "        'b':r\"I( strongly|'d like to| want to| would like to|) (encourage|recommend|demand|suggest|advocate|ask|implore|request|urge) (the Federal Communications Commission|Chairman Pai|the commission|the commissioners|the FCC|you|the government|Ajit Pai|) to (undo|repeal|rescind|overturn|reverse|) (Barack Obama's|President Obama's|The previous administration's|Obama's|The Obama\\/Wheeler|Tom Wheeler's|) (order|plan|scheme|power grab|decision|policy) to (take over|regulate|control|) (the Internet|Internet access|broadband|the web)\\.\",\n",
    "        'c':r\"(Individual Americans|Individuals|Americans|Citizens|Internet users|Individual citizens|People like me|), (rather than|as opposed to|not) (so-called experts|the FCC|big government|unelected bureaucrats|Washington|Washington bureaucrats|the FCC Enforcement Bureau|), ((deserve|should be empowered|should be able|should be free|ought) to|(should)) (enjoy|use|purchase|buy|select) (whatever|whichever|the|which) (services|products|applications) (we|they) (prefer|want|choose|desire)\\.\",\n",
    "        'd':r\"(Barack Obama's|President Obama's|The previous administration's|Obama's|The Obama\\/Wheeler|Tom Wheeler's|) (order|plan|scheme|power grab|decision|policy) to (take over|regulate|control|) (the Internet|Internet access|broadband|the web) is a (corruption|betrayal|perversion|exploitation |distortion) of (the open Internet|net neutrality)\\.\",\n",
    "        'e':r\"It (reversed|stopped|disrupted|broke|ended|undid) a (hands-off|free-market|pro-consumer|market-based|light-touch) (framework|approach|system|policy) that (worked|functioned|performed) (remarkably|supremely|very|very, very|exceptionally|fabulously|) (smoothly|successfully|well) for (many years|a long time|decades|two decades) with (bipartisan|Republican and Democrat|nearly universal|both parties'|broad bipartisan) (approval|backing|consensus|support)\\.\"\n",
    "    },\n",
    "    {\n",
    "        'a':r\"(The FCC should safeguard Internet freedom by keeping the bright-line net neutrality protections in place and upholding Title II\\.|The FCC should ensure a fair and open Internet for all by opposing efforts to undermine net neutrality\\.|I’m calling on the FCC to stand up for net neutrality and safeguard Title II protections\\.|The FCC needs to stand up for Internet users like me and keep the net neutrality rules that are already in effect\\.|The FCC must protect the open Internet by maintaining net neutrality protections under Title II\\.|The FCC should stand up for Internet users by safeguarding net neutrality\\.|We need the FCC to defend the rights of millions of Internet users by upholding net neutrality protections\\.|As an Internet user, I’m asking the FCC to protect the net neutrality protections currently in place\\.)\",\n",
    "        'b':r\"The FCC should (reject|throw out) Chairman Ajit Pai’s (proposal|plan) to (give|hand) (the telecom giants|the (government-subsidized |)telecom giants|the (government-subsidized |)ISP monopolies) like ( |)(AT&T|Verizon|Comcast), ( |)(AT&T|Verizon|Comcast), and ( |)(AT&T|Verizon|Comcast) (free rein|the authority|the legal cover) to (engage in data discrimination|throttle whatever they please|create Internet fast lanes), stripping (Internet users|consumers|users) of the (necessary|meaningful|vital) (access and privacy|privacy and access) (rules|protections|safeguards|) we (demanded|worked for|fought for) and (so recently won|won just two years ago|just recently won)\\.\",\n",
    "        'c':r\"I’m (afraid of|worried about|concerned about) (a “pay-to-play” Internet where ISPs can charge more for certain websites|ISPs being allowed to discriminate against certain types of data or websites|creating a tiered Internet with “fast lanes” for certain sites or services) because ([Uu]sers will have fewer options and a less diverse Internet|ISPs could have too much power to determine what I can do online|the rich want to get richer, and you're enabling them if you slash current FCC rules|it further increases the power of the already monolithic corporations with monopolies on almost all of the ISP market)\\.\",\n",
    "        'd':r\"Thankfully, the (existing|current) (FCC regulations|Open Internet rules|net neutrality rules) (mean|ensure) that (ISP monopolies|Internet providers|ISPs) can’t (block or slow|slow or block) (customers’|our|users’|Internet users’|consumers’) (access|ability) to (see certain|certain) (web services|websites) or (create Internet “fast lanes”|engage in data discrimination) by charging (online services and websites|websites and online services|websites and online service) (money|more money) to reach (people|consumers|customers) faster\\.\",\n",
    "        'e':r\"(Thank you for keeping Title II net neutrality rules in place to protect Internet users like me\\.|I appreciate you maintaining Title II net neutrality rules and the rights of Internet users like me\\.|I urge you to keep Title II net neutrality in place, and safeguard Internet users like me\\.|Thanks for protecting Internet users like me by upholding the existing Title II net neutrality rules\\.)\"\n",
    "    }\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Processing Regex0...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/jeff/anaconda3/lib/python3.6/site-packages/ipykernel_launcher.py:7: UserWarning: This pattern has match groups. To actually get the groups, use str.extract.\n",
      "  import sys\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Processing Regex1...\n",
      "CPU times: user 4min 31s, sys: 56 ms, total: 4min 31s\n",
      "Wall time: 4min 31s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "regex_score_threshholds = [2, 2]\n",
    "for index, regex_dict in enumerate(regex_dicts):\n",
    "    print(f\"Processing Regex{index}...\")\n",
    "    ## process the next regex list\n",
    "    cols = []\n",
    "    for col, regex in regex_dict.items():\n",
    "        cols.append(data['text_data'].str.contains(regex))\n",
    "    df_masks = (pd.concat(cols, axis=1).sum(axis=1) > regex_score_threshholds[index])\n",
    "    next_clusterid = data['level_0_cluster'].max()+1\n",
    "    data.loc[df_masks,'level_0_cluster'] = next_clusterid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>docid</th>\n",
       "      <th>text_data</th>\n",
       "      <th>dupe_count</th>\n",
       "      <th>level_0_cluster</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2628575</th>\n",
       "      <td>10379194</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked FaceTime, T...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2353851</th>\n",
       "      <td>9005702</td>\n",
       "      <td>Net Neutrality is important to me because without it, I may not be able to communicate with my friends and go on sites I enjoy. This whole ending Net Neutrality thing is bullshit.</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2451452</th>\n",
       "      <td>10132438</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked FaceTime, T...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>712851</th>\n",
       "      <td>2581090</td>\n",
       "      <td>I am in favor of strong net neutrality under Title II of the Telecommunications Act.\\n\\n\\nSincerely,\\nThomas Shaw</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1403315</th>\n",
       "      <td>3939737</td>\n",
       "      <td>Chairman Pai:  Hi, I'd like to comment on Net Neutrality and Title II. I want to ask the Federal Communications Commission to undo Barack Obama's order to control the web. Internet users, not so-called experts, should buy which applications they choose. Barack Obama's order to control the web is a distortion of the open Internet. It stopped a hands-off approach that worked exceptionally smoothly for two decades with broad bipartisan consensus.</td>\n",
       "      <td>2.0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2429466</th>\n",
       "      <td>10102096</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\r\\n\\r\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\r\\n\\r\\nNow is not the time to let giant ISPs censor what we see and do online.\\r\\n\\r\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40919</th>\n",
       "      <td>95227</td>\n",
       "      <td>ISPs must remain under Title 2. Asking companies to obey the spirit of Net Neutrality without having a system of law to enforce Net Neutrality is a meaningless gesture that will negatively impact the American people. Infrastructure and ISPs in many areas of the country are natural monopolies, and when people can't realistically choose a vital service, then it is up to the government to ensure people are not taken advantage of by the providers of that vital service. The internet must remain a...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>334318</th>\n",
       "      <td>742599</td>\n",
       "      <td>you must preserve net neutrality and protect title 9. Open internet is a basic right for all of us. We need to be the standard setters for the world.</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>544851</th>\n",
       "      <td>2200226</td>\n",
       "      <td>FCC:  I have concerns about net neutrality. I strongly implore the commissioners to repeal Tom Wheeler's policy to regulate the Internet. People like me, not unelected bureaucrats, should be able to buy whichever products they want. Tom Wheeler's policy to regulate the Internet is a betrayal of the open Internet. It stopped a free-market framework that functioned exceptionally successfully for two decades with bipartisan consensus.</td>\n",
       "      <td>2.0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>65194</th>\n",
       "      <td>146327</td>\n",
       "      <td>Preserve net neutrality under Title II regulation, please.</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>674164</th>\n",
       "      <td>2469777</td>\n",
       "      <td>Dear Chairman Pai,  I'm concerned about Internet Freedom. I would like to ask the Federal Communications Commission to repeal Barack Obama's policy to control the Internet. Americans, not the FCC Enforcement Bureau, should be free to purchase the services we want. Barack Obama's policy to control the Internet is a distortion of net neutrality. It ended a light-touch framework that worked fabulously successfully for a long time with broad bipartisan backing.</td>\n",
       "      <td>2.0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1579934</th>\n",
       "      <td>4864467</td>\n",
       "      <td>Dear Commissioners:  I want to give my opinion on network neutrality regulations. I want to implore the government to overturn The previous administration's policy to control the web. Individuals, not unelected bureaucrats, deserve to use which products we prefer. The previous administration's policy to control the web is a betrayal of net neutrality. It undid a pro-consumer system that functioned supremely smoothly for many years with Republican and Democrat backing.</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2096888</th>\n",
       "      <td>7870574</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked FaceTime, T...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2624765</th>\n",
       "      <td>10373806</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked FaceTime, T...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>827692</th>\n",
       "      <td>2753054</td>\n",
       "      <td>To the FCC:  With respect to net neutrality and Title II. I strongly suggest the FCC to undo Barack Obama's order to regulate the web. Individual Americans, not the FCC Enforcement Bureau, should be able to select the applications we desire. Barack Obama's order to regulate the web is a betrayal of net neutrality. It broke a light-touch framework that performed very, very successfully for many years with nearly universal support.</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>937242</th>\n",
       "      <td>2862640</td>\n",
       "      <td>Dear FCC,  Regarding Title 2 and net neutrality. I recommend the commission to overturn The previous administration's decision to regulate Internet access. Americans, rather than the FCC Enforcement Bureau, ought to buy whichever products they prefer. The previous administration's decision to regulate Internet access is a distortion of net neutrality. It disrupted a hands-off approach that worked exceptionally successfully for a long time with both parties' consensus.</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2556905</th>\n",
       "      <td>10279375</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked FaceTime, T...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49633</th>\n",
       "      <td>123169</td>\n",
       "      <td>The current rhetoric from the FCC chairman in regards to lightening regulations on ISPs is highly disturbing.  Net neutrality is a principle which must continue to guide the Internet.  All internet traffic, much like all speech, must be respected and given equal access.  Corporations must not be allowed to use their status as ISPs to extort others simply to receive equal service.</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2623570</th>\n",
       "      <td>10372190</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\r\\n\\r\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\r\\n\\r\\nNow is not the time to let giant ISPs censor what we see and do online.\\r\\n\\r\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked...</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1325132</th>\n",
       "      <td>3463189</td>\n",
       "      <td>To the Federal Communications Commission:  I would like to comment on Internet freedom. I would like to advocate the government to undo Tom Wheeler's power grab to regulate the Internet. Citizens, as opposed to big government, should be empowered to purchase whatever products we prefer. Tom Wheeler's power grab to regulate the Internet is a distortion of net neutrality. It broke a market-based approach that functioned very, very well for decades with broad bipartisan support.</td>\n",
       "      <td>2.0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>285357</th>\n",
       "      <td>601703</td>\n",
       "      <td>Protect net neutrality Andy persevere title two!</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1875081</th>\n",
       "      <td>7434235</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\r\\n\\r\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\r\\n\\r\\nNow is not the time to let giant ISPs censor what we see and do online.\\r\\n\\r\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1901089</th>\n",
       "      <td>7514842</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked FaceTime, T...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1404298</th>\n",
       "      <td>3941062</td>\n",
       "      <td>Dear Chairman Pai,  I am concerned about the FCC's Open Internet order. I'd like to demand Ajit Pai to overturn Obama's policy to control broadband. Americans, rather than unelected bureaucrats, should be free to select whichever services they want. Obama's policy to control broadband is a distortion of the open Internet. It disrupted a hands-off approach that functioned very, very smoothly for decades with bipartisan backing.</td>\n",
       "      <td>2.0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>267469</th>\n",
       "      <td>500802</td>\n",
       "      <td>As an educator we must not allow for sites to have differing speeds. All things on the internet should come in at the same speed across carriers and platforms and site. Things have gotten better over the last few years and we should be able to increase speed over the internet not slowing it down.</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2336986</th>\n",
       "      <td>8839774</td>\n",
       "      <td>The internet is literally the best thing to happen to freedom of information since the printing press. Don't let it become restricted by the interests of corporations, or block access for people that can't pay. Money should have no bearing on access to information.</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>873095</th>\n",
       "      <td>2798474</td>\n",
       "      <td>Dear Chairman Pai,  In reference to Title II rules. I strongly urge you to overturn The Obama/Wheeler decision to control the web. Individual Americans, not the FCC Enforcement Bureau, should be free to buy which products they want. The Obama/Wheeler decision to control the web is a betrayal of net neutrality. It broke a hands-off approach that functioned exceptionally well for decades with Republican and Democrat approval.</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2594541</th>\n",
       "      <td>10331801</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked FaceTime, T...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>791126</th>\n",
       "      <td>2716488</td>\n",
       "      <td>Dear Mr. Pai,  I am concerned about the Open Internet order. I strongly recommend the commissioners to undo President Obama's scheme to take over the Internet. Individual Americans, not big government, ought to enjoy the applications we choose. President Obama's scheme to take over the Internet is a betrayal of the open Internet. It disrupted a free-market framework that performed exceptionally smoothly for two decades with broad bipartisan backing.</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2161444</th>\n",
       "      <td>8017178</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked FaceTime, T...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>488295</th>\n",
       "      <td>1781302</td>\n",
       "      <td>I support an open internet. Do not allow these changes.</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2417220</th>\n",
       "      <td>9516060</td>\n",
       "      <td>The FCC is the correct regulatory division for broadband internet service because at this point society relies so heavily on internet connectivity, that broadband internet is closer to a telecommunications utility service than a business, and should be regulated as such. Companies should not have the ability to leverage the geographic monopolies that they acquired during the period where the FTC was the regulatory body to extort customers and businesses by throttling or fully withholding int...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2265472</th>\n",
       "      <td>8219019</td>\n",
       "      <td>The FCC must protect the open Internet by maintaining net neutrality protections under Title II. \\r\\n\\r\\nThe FCC should throw out Chairman Ajit Pai’s plan to hand the government-subsidized ISP monopolies like AT&amp;T, Comcast, and Verizon the legal cover to engage in data discrimination, stripping users of the necessary privacy and access safeguards we demanded and just recently won. \\r\\n\\r\\nI’m afraid of a “pay-to-play” Internet where ISPs can charge more for certain websites because users wil...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>156365</th>\n",
       "      <td>278090</td>\n",
       "      <td>I support Net Neutrality - I do not support any change to the current law</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>295754</th>\n",
       "      <td>629685</td>\n",
       "      <td>I strongly support Net Neutrality backed by Title II oversight. Internet Service Providers absolutely should be classified under Title II regulation. ISP's should not be allowed to throttle internet content based on type. Please leave Title II alone.</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1169930</th>\n",
       "      <td>3272535</td>\n",
       "      <td>Dear Mr. Pai,  I have concerns about the FCC's so-called Open Internet order. I would like to recommend the commission to undo The previous administration's power grab to take over broadband. Internet users, rather than big government, should purchase the applications they choose. The previous administration's power grab to take over broadband is a betrayal of net neutrality. It reversed a pro-consumer policy that performed exceptionally successfully for two decades with bipartisan approval.</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1903540</th>\n",
       "      <td>7522452</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked FaceTime, T...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1243109</th>\n",
       "      <td>3345717</td>\n",
       "      <td>Dear Commissioners:  I'd like to share my thoughts on Internet regulation. I want to recommend the FCC to undo Tom Wheeler's power grab to take over Internet access. Individual citizens, not so-called experts, ought to use whatever services we prefer. Tom Wheeler's power grab to take over Internet access is a distortion of the open Internet. It stopped a hands-off approach that performed very, very successfully for many years with broad bipartisan support.</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>909077</th>\n",
       "      <td>2834475</td>\n",
       "      <td>Dear FCC,  My comments re: the Open Internet order. I'd like to ask Chairman Pai to rescind The previous administration's plan to regulate broadband. Individual Americans, not Washington bureaucrats, deserve to select which services they prefer. The previous administration's plan to regulate broadband is a perversion of net neutrality. It reversed a market-based policy that worked remarkably successfully for many years with both parties' approval.</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2904458</th>\n",
       "      <td>18164602</td>\n",
       "      <td>I am in favor of strong net neutrality under Title II of the Telecommunications Act.\\n\\n\\nSincerely,\\nJoel Lewis</td>\n",
       "      <td>201.0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>988513</th>\n",
       "      <td>2913916</td>\n",
       "      <td>To whom it may concern:  Hi, I'd like to comment on net neutrality rules. I'd like to encourage Chairman Pai to undo Barack Obama's order to take over the Internet. Individual citizens, as opposed to the FCC Enforcement Bureau, should be free to buy the applications they desire. Barack Obama's order to take over the Internet is a exploitation  of net neutrality. It broke a free-market approach that functioned exceptionally successfully for two decades with bipartisan consensus.</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1584387</th>\n",
       "      <td>4884050</td>\n",
       "      <td>Dear FCC,  With respect to Net neutrality. I'd like to implore the commission to reverse The previous administration's policy to regulate the web. Individuals, not big government, deserve to select which applications we want. The previous administration's policy to regulate the web is a distortion of the open Internet. It stopped a market-based system that functioned remarkably smoothly for a long time with both parties' consensus.</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2172016</th>\n",
       "      <td>8035871</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked FaceTime, T...</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2109182</th>\n",
       "      <td>7897873</td>\n",
       "      <td>I’m calling on the FCC to stand up for net neutrality and safeguard Title II protections. \\r\\n\\r\\nThe FCC should reject Chairman Ajit Pai’s proposal to hand the government-subsidized telecom giants like Verizon, Comcast, and AT&amp;T the legal cover to create Internet fast lanes, stripping Internet users of the vital privacy and access rules we demanded and so recently won. \\r\\n\\r\\nI’m worried about creating a tiered Internet with “fast lanes” for certain sites or services because users will hav...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2250251</th>\n",
       "      <td>8165130</td>\n",
       "      <td>The FCC, as an arm of government, should first and foremost be upholding the Constitutional rights of the citizenry to have open communication unrestricted by the whims and profit motive of corporate internet and media giants.  Please use your good offices to safeguard Net Neutrality for all.</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2787982</th>\n",
       "      <td>10931848</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked FaceTime, T...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>913355</th>\n",
       "      <td>2838753</td>\n",
       "      <td>Chairman Pai:  Hi, I'd like to comment on the FCC's so-called Open Internet order. I advocate the FCC to repeal President Obama's order to regulate broadband. Individual Americans, not Washington, should be empowered to use the products we want. President Obama's order to regulate broadband is a distortion of net neutrality. It broke a hands-off approach that worked remarkably smoothly for many years with nearly universal consensus.</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2217763</th>\n",
       "      <td>8122759</td>\n",
       "      <td>The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&amp;T blocked FaceTime, T...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1645722</th>\n",
       "      <td>5010525</td>\n",
       "      <td>I expect the FCC to protect my free and quick access to *my* preferred content on the Internet.</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2391998</th>\n",
       "      <td>9196770</td>\n",
       "      <td>As an Internet user, I'm asking the FCC to protect the net neutrality protections currently in place. \\n\\nThe FCC should throw out Chairman Ajit Pai's proposal to give the ISP monopolies like Comcast, AT&amp;T, and Verizon the authority to create Internet fast lanes, stripping Internet users of the meaningful access and privacy protections we fought for and just recently won. \\n\\nI'm concerned about ISPs being allowed to discriminate against certain types of data or websites, because users will ...</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            docid  \\\n",
       "2628575  10379194   \n",
       "2353851   9005702   \n",
       "2451452  10132438   \n",
       "712851    2581090   \n",
       "1403315   3939737   \n",
       "2429466  10102096   \n",
       "40919       95227   \n",
       "334318     742599   \n",
       "544851    2200226   \n",
       "65194      146327   \n",
       "674164    2469777   \n",
       "1579934   4864467   \n",
       "2096888   7870574   \n",
       "2624765  10373806   \n",
       "827692    2753054   \n",
       "937242    2862640   \n",
       "2556905  10279375   \n",
       "49633      123169   \n",
       "2623570  10372190   \n",
       "1325132   3463189   \n",
       "285357     601703   \n",
       "1875081   7434235   \n",
       "1901089   7514842   \n",
       "1404298   3941062   \n",
       "267469     500802   \n",
       "2336986   8839774   \n",
       "873095    2798474   \n",
       "2594541  10331801   \n",
       "791126    2716488   \n",
       "2161444   8017178   \n",
       "488295    1781302   \n",
       "2417220   9516060   \n",
       "2265472   8219019   \n",
       "156365     278090   \n",
       "295754     629685   \n",
       "1169930   3272535   \n",
       "1903540   7522452   \n",
       "1243109   3345717   \n",
       "909077    2834475   \n",
       "2904458  18164602   \n",
       "988513    2913916   \n",
       "1584387   4884050   \n",
       "2172016   8035871   \n",
       "2109182   7897873   \n",
       "2250251   8165130   \n",
       "2787982  10931848   \n",
       "913355    2838753   \n",
       "2217763   8122759   \n",
       "1645722   5010525   \n",
       "2391998   9196770   \n",
       "\n",
       "                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   text_data  \\\n",
       "2628575  The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked FaceTime, T...   \n",
       "2353851                                                                                                                                                                                                                                                                                                                                  Net Neutrality is important to me because without it, I may not be able to communicate with my friends and go on sites I enjoy. This whole ending Net Neutrality thing is bullshit.   \n",
       "2451452  The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked FaceTime, T...   \n",
       "712851                                                                                                                                                                                                                                                                                                                                                                                                     I am in favor of strong net neutrality under Title II of the Telecommunications Act.\\n\\n\\nSincerely,\\nThomas Shaw   \n",
       "1403315                                                      Chairman Pai:  Hi, I'd like to comment on Net Neutrality and Title II. I want to ask the Federal Communications Commission to undo Barack Obama's order to control the web. Internet users, not so-called experts, should buy which applications they choose. Barack Obama's order to control the web is a distortion of the open Internet. It stopped a hands-off approach that worked exceptionally smoothly for two decades with broad bipartisan consensus.   \n",
       "2429466  The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\r\\n\\r\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\r\\n\\r\\nNow is not the time to let giant ISPs censor what we see and do online.\\r\\n\\r\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked...   \n",
       "40919    ISPs must remain under Title 2. Asking companies to obey the spirit of Net Neutrality without having a system of law to enforce Net Neutrality is a meaningless gesture that will negatively impact the American people. Infrastructure and ISPs in many areas of the country are natural monopolies, and when people can't realistically choose a vital service, then it is up to the government to ensure people are not taken advantage of by the providers of that vital service. The internet must remain a...   \n",
       "334318                                                                                                                                                                                                                                                                                                                                                                 you must preserve net neutrality and protect title 9. Open internet is a basic right for all of us. We need to be the standard setters for the world.   \n",
       "544851                                                                   FCC:  I have concerns about net neutrality. I strongly implore the commissioners to repeal Tom Wheeler's policy to regulate the Internet. People like me, not unelected bureaucrats, should be able to buy whichever products they want. Tom Wheeler's policy to regulate the Internet is a betrayal of the open Internet. It stopped a free-market framework that functioned exceptionally successfully for two decades with bipartisan consensus.   \n",
       "65194                                                                                                                                                                                                                                                                                                                                                                                                                                                             Preserve net neutrality under Title II regulation, please.   \n",
       "674164                                         Dear Chairman Pai,  I'm concerned about Internet Freedom. I would like to ask the Federal Communications Commission to repeal Barack Obama's policy to control the Internet. Americans, not the FCC Enforcement Bureau, should be free to purchase the services we want. Barack Obama's policy to control the Internet is a distortion of net neutrality. It ended a light-touch framework that worked fabulously successfully for a long time with broad bipartisan backing.   \n",
       "1579934                             Dear Commissioners:  I want to give my opinion on network neutrality regulations. I want to implore the government to overturn The previous administration's policy to control the web. Individuals, not unelected bureaucrats, deserve to use which products we prefer. The previous administration's policy to control the web is a betrayal of net neutrality. It undid a pro-consumer system that functioned supremely smoothly for many years with Republican and Democrat backing.   \n",
       "2096888  The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked FaceTime, T...   \n",
       "2624765  The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked FaceTime, T...   \n",
       "827692                                                                     To the FCC:  With respect to net neutrality and Title II. I strongly suggest the FCC to undo Barack Obama's order to regulate the web. Individual Americans, not the FCC Enforcement Bureau, should be able to select the applications we desire. Barack Obama's order to regulate the web is a betrayal of net neutrality. It broke a light-touch framework that performed very, very successfully for many years with nearly universal support.   \n",
       "937242                              Dear FCC,  Regarding Title 2 and net neutrality. I recommend the commission to overturn The previous administration's decision to regulate Internet access. Americans, rather than the FCC Enforcement Bureau, ought to buy whichever products they prefer. The previous administration's decision to regulate Internet access is a distortion of net neutrality. It disrupted a hands-off approach that worked exceptionally successfully for a long time with both parties' consensus.   \n",
       "2556905  The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked FaceTime, T...   \n",
       "49633                                                                                                                         The current rhetoric from the FCC chairman in regards to lightening regulations on ISPs is highly disturbing.  Net neutrality is a principle which must continue to guide the Internet.  All internet traffic, much like all speech, must be respected and given equal access.  Corporations must not be allowed to use their status as ISPs to extort others simply to receive equal service.   \n",
       "2623570  The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\r\\n\\r\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\r\\n\\r\\nNow is not the time to let giant ISPs censor what we see and do online.\\r\\n\\r\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked...   \n",
       "1325132                     To the Federal Communications Commission:  I would like to comment on Internet freedom. I would like to advocate the government to undo Tom Wheeler's power grab to regulate the Internet. Citizens, as opposed to big government, should be empowered to purchase whatever products we prefer. Tom Wheeler's power grab to regulate the Internet is a distortion of net neutrality. It broke a market-based approach that functioned very, very well for decades with broad bipartisan support.   \n",
       "285357                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Protect net neutrality Andy persevere title two!   \n",
       "1875081  The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\r\\n\\r\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\r\\n\\r\\nNow is not the time to let giant ISPs censor what we see and do online.\\r\\n\\r\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked...   \n",
       "1901089  The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked FaceTime, T...   \n",
       "1404298                                                                       Dear Chairman Pai,  I am concerned about the FCC's Open Internet order. I'd like to demand Ajit Pai to overturn Obama's policy to control broadband. Americans, rather than unelected bureaucrats, should be free to select whichever services they want. Obama's policy to control broadband is a distortion of the open Internet. It disrupted a hands-off approach that functioned very, very smoothly for decades with bipartisan backing.   \n",
       "267469                                                                                                                                                                                                             As an educator we must not allow for sites to have differing speeds. All things on the internet should come in at the same speed across carriers and platforms and site. Things have gotten better over the last few years and we should be able to increase speed over the internet not slowing it down.   \n",
       "2336986                                                                                                                                                                                                                                            The internet is literally the best thing to happen to freedom of information since the printing press. Don't let it become restricted by the interests of corporations, or block access for people that can't pay. Money should have no bearing on access to information.   \n",
       "873095                                                                           Dear Chairman Pai,  In reference to Title II rules. I strongly urge you to overturn The Obama/Wheeler decision to control the web. Individual Americans, not the FCC Enforcement Bureau, should be free to buy which products they want. The Obama/Wheeler decision to control the web is a betrayal of net neutrality. It broke a hands-off approach that functioned exceptionally well for decades with Republican and Democrat approval.   \n",
       "2594541  The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked FaceTime, T...   \n",
       "791126                                                 Dear Mr. Pai,  I am concerned about the Open Internet order. I strongly recommend the commissioners to undo President Obama's scheme to take over the Internet. Individual Americans, not big government, ought to enjoy the applications we choose. President Obama's scheme to take over the Internet is a betrayal of the open Internet. It disrupted a free-market framework that performed exceptionally smoothly for two decades with broad bipartisan backing.   \n",
       "2161444  The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked FaceTime, T...   \n",
       "488295                                                                                                                                                                                                                                                                                                                                                                                                                                                               I support an open internet. Do not allow these changes.   \n",
       "2417220  The FCC is the correct regulatory division for broadband internet service because at this point society relies so heavily on internet connectivity, that broadband internet is closer to a telecommunications utility service than a business, and should be regulated as such. Companies should not have the ability to leverage the geographic monopolies that they acquired during the period where the FTC was the regulatory body to extort customers and businesses by throttling or fully withholding int...   \n",
       "2265472  The FCC must protect the open Internet by maintaining net neutrality protections under Title II. \\r\\n\\r\\nThe FCC should throw out Chairman Ajit Pai’s plan to hand the government-subsidized ISP monopolies like AT&T, Comcast, and Verizon the legal cover to engage in data discrimination, stripping users of the necessary privacy and access safeguards we demanded and just recently won. \\r\\n\\r\\nI’m afraid of a “pay-to-play” Internet where ISPs can charge more for certain websites because users wil...   \n",
       "156365                                                                                                                                                                                                                                                                                                                                                                                                                                             I support Net Neutrality - I do not support any change to the current law   \n",
       "295754                                                                                                                                                                                                                                                            I strongly support Net Neutrality backed by Title II oversight. Internet Service Providers absolutely should be classified under Title II regulation. ISP's should not be allowed to throttle internet content based on type. Please leave Title II alone.   \n",
       "1169930     Dear Mr. Pai,  I have concerns about the FCC's so-called Open Internet order. I would like to recommend the commission to undo The previous administration's power grab to take over broadband. Internet users, rather than big government, should purchase the applications they choose. The previous administration's power grab to take over broadband is a betrayal of net neutrality. It reversed a pro-consumer policy that performed exceptionally successfully for two decades with bipartisan approval.   \n",
       "1903540  The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked FaceTime, T...   \n",
       "1243109                                         Dear Commissioners:  I'd like to share my thoughts on Internet regulation. I want to recommend the FCC to undo Tom Wheeler's power grab to take over Internet access. Individual citizens, not so-called experts, ought to use whatever services we prefer. Tom Wheeler's power grab to take over Internet access is a distortion of the open Internet. It stopped a hands-off approach that performed very, very successfully for many years with broad bipartisan support.   \n",
       "909077                                                   Dear FCC,  My comments re: the Open Internet order. I'd like to ask Chairman Pai to rescind The previous administration's plan to regulate broadband. Individual Americans, not Washington bureaucrats, deserve to select which services they prefer. The previous administration's plan to regulate broadband is a perversion of net neutrality. It reversed a market-based policy that worked remarkably successfully for many years with both parties' approval.   \n",
       "2904458                                                                                                                                                                                                                                                                                                                                                                                                     I am in favor of strong net neutrality under Title II of the Telecommunications Act.\\n\\n\\nSincerely,\\nJoel Lewis   \n",
       "988513                    To whom it may concern:  Hi, I'd like to comment on net neutrality rules. I'd like to encourage Chairman Pai to undo Barack Obama's order to take over the Internet. Individual citizens, as opposed to the FCC Enforcement Bureau, should be free to buy the applications they desire. Barack Obama's order to take over the Internet is a exploitation  of net neutrality. It broke a free-market approach that functioned exceptionally successfully for two decades with bipartisan consensus.   \n",
       "1584387                                                                  Dear FCC,  With respect to Net neutrality. I'd like to implore the commission to reverse The previous administration's policy to regulate the web. Individuals, not big government, deserve to select which applications we want. The previous administration's policy to regulate the web is a distortion of the open Internet. It stopped a market-based system that functioned remarkably smoothly for a long time with both parties' consensus.   \n",
       "2172016  The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked FaceTime, T...   \n",
       "2109182  I’m calling on the FCC to stand up for net neutrality and safeguard Title II protections. \\r\\n\\r\\nThe FCC should reject Chairman Ajit Pai’s proposal to hand the government-subsidized telecom giants like Verizon, Comcast, and AT&T the legal cover to create Internet fast lanes, stripping Internet users of the vital privacy and access rules we demanded and so recently won. \\r\\n\\r\\nI’m worried about creating a tiered Internet with “fast lanes” for certain sites or services because users will hav...   \n",
       "2250251                                                                                                                                                                                                                The FCC, as an arm of government, should first and foremost be upholding the Constitutional rights of the citizenry to have open communication unrestricted by the whims and profit motive of corporate internet and media giants.  Please use your good offices to safeguard Net Neutrality for all.   \n",
       "2787982  The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked FaceTime, T...   \n",
       "913355                                                                  Chairman Pai:  Hi, I'd like to comment on the FCC's so-called Open Internet order. I advocate the FCC to repeal President Obama's order to regulate broadband. Individual Americans, not Washington, should be empowered to use the products we want. President Obama's order to regulate broadband is a distortion of net neutrality. It broke a hands-off approach that worked remarkably smoothly for many years with nearly universal consensus.   \n",
       "2217763  The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to protect them.\\n\\nI don't want ISPs to have the power to block websites, slow them down, give some sites an advantage over others, or split the Internet into \"fast lanes\" for companies that pay and \"slow lanes\" for the rest.\\n\\nNow is not the time to let giant ISPs censor what we see and do online.\\n\\nCensorship by ISPs is a serious problem. Comcast has throttled Netflix, AT&T blocked FaceTime, T...   \n",
       "1645722                                                                                                                                                                                                                                                                                                                                                                                                                      I expect the FCC to protect my free and quick access to *my* preferred content on the Internet.   \n",
       "2391998  As an Internet user, I'm asking the FCC to protect the net neutrality protections currently in place. \\n\\nThe FCC should throw out Chairman Ajit Pai's proposal to give the ISP monopolies like Comcast, AT&T, and Verizon the authority to create Internet fast lanes, stripping Internet users of the meaningful access and privacy protections we fought for and just recently won. \\n\\nI'm concerned about ISPs being allowed to discriminate against certain types of data or websites, because users will ...   \n",
       "\n",
       "         dupe_count  level_0_cluster  \n",
       "2628575         1.0                0  \n",
       "2353851         1.0               -1  \n",
       "2451452         1.0                0  \n",
       "712851          1.0                2  \n",
       "1403315         2.0               13  \n",
       "2429466         1.0                0  \n",
       "40919           1.0               -1  \n",
       "334318          1.0               -1  \n",
       "544851          2.0               13  \n",
       "65194           1.0               -1  \n",
       "674164          2.0               13  \n",
       "1579934         1.0               13  \n",
       "2096888         1.0                0  \n",
       "2624765         1.0                0  \n",
       "827692          1.0               13  \n",
       "937242          1.0               13  \n",
       "2556905         1.0                0  \n",
       "49633           1.0               -1  \n",
       "2623570         3.0                0  \n",
       "1325132         2.0               13  \n",
       "285357          1.0               -1  \n",
       "1875081         1.0                0  \n",
       "1901089         1.0                0  \n",
       "1404298         2.0               13  \n",
       "267469          1.0               -1  \n",
       "2336986         1.0               -1  \n",
       "873095          1.0               13  \n",
       "2594541         1.0                0  \n",
       "791126          1.0               13  \n",
       "2161444         1.0                0  \n",
       "488295          1.0               -1  \n",
       "2417220         1.0               -1  \n",
       "2265472         1.0               14  \n",
       "156365          1.0               -1  \n",
       "295754          1.0               -1  \n",
       "1169930         1.0               13  \n",
       "1903540         1.0                0  \n",
       "1243109         1.0               13  \n",
       "909077          1.0               13  \n",
       "2904458       201.0                2  \n",
       "988513          1.0               13  \n",
       "1584387         1.0               13  \n",
       "2172016         2.0                0  \n",
       "2109182         1.0               14  \n",
       "2250251         1.0               -1  \n",
       "2787982         1.0                0  \n",
       "913355          1.0               13  \n",
       "2217763         1.0                0  \n",
       "1645722         1.0               -1  \n",
       "2391998         2.0                1  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#take another peek\n",
    "data.sample(50)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "level_0_cluster\n",
       "-1     11033922.0\n",
       " 0      1571457.0\n",
       " 1       166295.0\n",
       " 2      7542184.0\n",
       " 3         5068.0\n",
       " 4         8591.0\n",
       " 5        18922.0\n",
       " 6         5175.0\n",
       " 7          526.0\n",
       " 8        47642.0\n",
       " 9          237.0\n",
       " 10         793.0\n",
       " 11      169987.0\n",
       " 12       20904.0\n",
       " 13     1384868.0\n",
       " 14      102285.0\n",
       "Name: dupe_count, dtype: float64"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#what's the document count for each cluster? \n",
    "doc_counts = data.groupby('level_0_cluster').sum()['dupe_count']\n",
    "doc_counts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#save to file so we can iterate it back to EDA or proceed to HDBSCAN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "data.to_csv('proc_17_108_unique_comments_text_dupe_count_leveled.csv', index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}