Auto-login to web proxies using NetworkManager

My ISP uses a web proxy that one has to log into to access the Internet. This logging in is a manual, repetitive process, which is easily automatable. So I embarked on a few hour-long project to get to the proxy, supply login credentials and configure NetworkManager to auto-login via running the script each time a connection goes up.

It’s not just ISPs — hotel wifi networks, airport wifis, all use such web-based proxies that one has to login to first before the ‘net becomes accessible. So the steps I followed can be easily followed by others to add support for auto-logging into such web proxies.

I’ll get to the details in a bit, but I’ll first point to the code (licensed under the GPL, v2). It’s written in Python, a language that’s relatively new for me. I’ve written a couple of small programs earlier, but those were just enough to remind me of the syntax; I had to frequently look up the Python docs to get a lot of the details, like interacting with http servers, cookie management, config file management and so on. My C-style writing of the Python script might be evident: it should be possible for someone with more experience in Python to shorten or optimise the script.

My ISP, Tikona Digital Networks, uses a somewhat roundabout way to bring up the login page: for any URL accessed before the proxy login, it first displays an http page that has a redirect URL and a ‘Please wait while login page is loaded’ message. The page to be redirected to is then loaded. This page shows another ‘Please wait’ message, sets a cookie and does a POST action to the real login page after a 5-second timeout. The real login page asks for the username and password. After providing that info, one has to click on the Login button, which translates to a javascript-based POST request, and if the username/password provided match the ones in their database, we’re authenticated to the web proxy. The web proxy doesn’t interfere with any further ‘net access.

Now that I’ve gone through the rough overview of the approach to take, I’ll detail the steps I took to get this script ready:

Step 1: Follow the redirect URL

Open a browser, type in some URL — say ‘www.google.com’. This always resulted in a page that asked me to wait while it went to the login page.

OK, so time for a short python script to check what’s happening:

import urllib

f = urllib.urlopen("http://www.google.com")
s = f.read()
f.close()

print s

This snippet accesses the google.com website and dumps on the screen the result of the http request.

Here’s the dump that I get before the login.

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<title>Please wait while the login page is loaded...</title>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<META HTTP-EQUIV="PRAGMA" CONTENT="NO-CACHE"/>
<META HTTP-EQUIV="EXPIRES" CONTENT="-1"/>
<META HTTP-EQUIV="Refresh" CONTENT="2;URL=https://login.tikona.in/userportal/?requesturi=http%3a%2f%2fgoogle%2ecom%2f&ip=113%2e193%2e150%2e95&nas=tikonapune&requestip=google%2ecom&sc=5a54aa1fd2de7a9c2b92a865de55b943">
</head>
<body>
<p align="center">Please wait...<p>
Please wait while the login page is loaded...
<!---
<msc>
<login_url><![CDATA[https://login.tikona.in/userportal/NSCLOGIN.do?requesturi=http%3a%2f%2fgoogle%2ecom%2f&ip=113%2e193%2e150%2e95&mac=00%3a16%3a01%3a8e%3a06%3a92&nas=tikonapune&requestip=google%2ecom&sc=5a54aa1fd2de7a9c2b92a865de55b943]]></login_url>
<logout_url><![CDATA[https://login.tikona.in/userportal/NSCLOGOUT.do?requesturi=http%3a%2f%2fgoogle%2ecom%2f&ip=113%2e193%2e150%2e95&mac=00%3a16%3a01%3a8e%3a06%3a92&nas=tikonapune&requestip=google%2ecom&sc=5a54aa1fd2de7a9c2b92a865de55b943]]></logout_url>
<status_url><![CDATA[https://login.tikona.in/userportal/NSCSTATUS.do?requesturi=http%3a%2f%2fgoogle%2ecom%2f&ip=113%2e193%2e150%2e95&mac=00%3a16%3a01%3a8e%3a06%3a92&nas=tikonapune&requestip=google%2ecom&sc=5a54aa1fd2de7a9c2b92a865de55b943]]></status_url>
<update_url><![CDATA[https://login.tikona.in/userportal/NSCUPDATE.do?requesturi=http%3a%2f%2fgoogle%2ecom%2f&ip=113%2e193%2e150%2e95&mac=00%3a16%3a01%3a8e%3a06%3a92&nas=tikonapune&requestip=google%2ecom&sc=5a54aa1fd2de7a9c2b92a865de55b943]]></update_url>
<content_url><![CDATA[https://login.tikona.in/userportal/NSCCONTENT.do?requesturi=http%3a%2f%2fgoogle%2ecom%2f&ip=113%2e193%2e150%2e95&mac=00%3a16%3a01%3a8e%3a06%3a92&nas=tikonapune&requestip=google%2ecom&sc=5a54aa1fd2de7a9c2b92a865de55b943]]></content_url>
</msc>
-->

</body>
</html>

This shows there’s a redirect that’ll happen after the timeout (the META HTTP-EQUIV=”Refresh” line). The redirect is to the link shown.

Step 2: Get the redirect link

So now our task is to get the link from the http-equiv header and open that later. Using regular expressions, we can remove the text around the link and just obtain the link:

refresh_url_pattern = "HTTP-EQUIV="Refresh" CONTENT="2;URL=(.*)">"
refresh_url = search(refresh_url_pattern, s)

The URL to access is then available in refresh_url.group(1). group(1) contains the matched string in parentheses above in the pattern searched.

Now open the page obtained in the refresh URL:

f = urllib.urlopen(refresh_url.group(1))
s = f.read()

s now contains:

<html>
<head>
<title>Powered by Inventum</title>
<SCRIPT>
function moveToLogin() {
setTimeout("loadForm()",500);
}
function loadForm(){
document.forms[0].action="login.do?requesturi=http%3A%2F%2Fgoogle.com%2F&act=null";
document.forms[0].method="post";
document.forms[0].submit();
}
</SCRIPT>
</head>
<body onload="moveToLogin();">
<FORM>
Loading the login page...
</FORM>
</body>
</html>

Step 3: Get the base URL, open login page

So this page does an HTTP POST request. The URL of the new page being loaded is relative to the current one, so we have to extract the base URL from the previous redirect URL obtained.

base_url_pattern = "(http.*/)(?.*)$"
base_url = search(base_url_pattern, refresh_url.group(0))

The baseurl is then available via base_url.group(1). This regular expression pattern isolates the text before the first ‘?’, as is found in the refresh URL above.

So now we have to load the page login.do which is at address ‘https://login.tikona.in‘ and which is to be passed the parameters ‘?requesturi=http%3A%2F%2Fgoogle.com%2F&act=null‘. This calls for another regular expression by which we can isolate the ‘login.do…‘ part from the ‘action’ part of the POST request above.

load_form_pattern = ".*action="(.*)";"
load_form_id = search(load_form_pattern, s)
load_form_url = base_url.group(1) + load_form_id.group(1)

load_form_url is now the URL we need to access to get to the login page:

f = urllib.urlopen(load_form_url)
s = f.read()

This should get our login page.

But it’s not. After spending some time checking and double-checking what’s happening I couldn’t see anything going wrong. There was just one more thing to try: cookies. I disabled cookies in firefox and tried accessing the page. Voila, no login page.

Step 4: Enable cookie handling

So we now have to enable cookies in our python script to be able to enter login information. The urllib2 and cookielib libraries do that for us, so a slight re-write of the code gets us to this:

import urllib, urllib2, cookielib, ConfigParser, os
from re import search

cj = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))

f = opener.open("http://google.com")
s = f.read()

All other open calls (urllib.urlopen) are now replaced by opener.open. This way cookies are handled for the session and the login page appears after accessing the load_form_url:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<title>Tikona Digital Networks</title>
<link rel="stylesheet" type="text/css" href="/userportal/pages/css/style.css" />
<script language="JavaScript" src="/userportal/pages/js/cookie.js"></script>
<script language="JavaScript" src="/userportal/pages/js/common.js"></script>

</head>

<body>
<form name="form1">
<div id="wrap">
<div class="background_login">
<div class="logo_header">
<div class="logoimg"><img src="/userportal/pages/images/logo.jpg" alt="Tikona Digital Networks" /></div>
<div class="sitelink"><a href="http://www.tikona.in" target="_blank">www.tikona.in</a></div>

</div>
<div class="clear"></div>
<div class="login_box">
<div id="right_curved_block">
<div class="blue_head">
<div class="blue_head_right">
<div class="blue_head_left">&nbsp;</div>
<div class="hdng">Login</div>
</div>
</div>

<div class="clear"></div>
<div class="block_content">
<div class="form">
<table height="100%" border="0" cellpadding="0" cellspacing="0">

<tr>


<td width="126"><label>Service Type</label></td>
<td width="200" align="left" valign="middle">
<select name="type"><option value="1">Check Account Details</option>

<option value="2" selected="selected">Internet Access</option></select>
</td>

</tr>
<tr>
<td width="126"><label>User Name</label></td>
<td width="200" align="left" valign="middle"><input type="text" name="username" value="" class="logintext">

</td>
</tr>
<tr>
<td width="126"><label>Password</label></td>
<td width="200" align="left" valign="middle"><input type="password" name="password" value="" class="loginpassword"></td>
</tr>
<tr>
<td width="126"><label>Remember me</label></td>

<td width="200" align="left" valign="middle"><div style=" width:30%; float:left;"><input name="remeberme" id="rememberme" type="checkbox" class="checkbox"/></div>
<div style=" width:70%; float:right;"><a href="javascript:savesettings()"><img src="/userportal/pages/images/login.gif" alt="" width="117" height="30" hspace="0" vspace="0" border="0" align="right" /></a></div></td>
</tr>
</table>
</div>
</div>
<div class="clear"> </div>
<div class="white_bottom">
</div>
</div>
</div>

<div class="tips_box">
<div class="v_box">
<div id="tips_block">
<div class="white_head_v">
<div class="blue_head_right">
<div class="white_head_left_v">&nbsp;</div>
<div class="wbs_version">&nbsp;</div>
</div>
</div>
<div class="clear"></div>
<div class="block_content">
<div class="scrol">
<h1>Importance of Billing Account Number</h1>
<br />

<font size="2">
<ul>
<li>Billing Account Number (BAN) is a 9 digit unique identification number of your Tikona Wi-Bro service bill
account. It is mentioned below your name and address in the bill.</li>
<li>Bill payments done through cheque or demand draft should mandatorily have BAN mentioned on them. <br />
<span style="color:#558ed5">Example:</span> Cheque or demand draft should be drawn in the name of &lsquo;
Tikona Digital Networks Pvt. Ltd. a/c xxx xxx xxx&rsquo;. Here &lsquo;xxx xxx xxx&rsquo; denotes your BAN.
</li>
<li>If the BAN is not mentioned or incorrectly mentioned on the cheque or demand draft, the bill amount does
not get credited against your Tikona Wi-Bro service account.</li>
<li>In case you have paid bill through cheque or demand draft without mentioning BAN on it and the amount is
not credited to your Tikona billing account, then please contact TikonaCare at 1800 20 94276. Kindly furnish
your cheque number, service ID, BAN and bank statement for payment verification.</li>
</ul>
</font><br />
</div>
</div>
<div class="clear"> </div>
<div class="white_bottom">&nbsp;</div>
</div>
</div>

</div>
<div style="padding:110px 0 0 0; float:left; width:100%;">
<div class="helpline">
Tikona Care: 1800 20 94276 | <a href="mailto:customercare@tikona.in">customercare@tikona.in</a></div>
</div>

<div class="footer_line">&nbsp;</div>
<div class="footer_blueline"></div>

<div class="footer">
Copyright &copy; 2009. Tikona Digital Networks. All right Reserved.
</div>
</div>
</div>
<input type="hidden" name="act" value="null">
</form>
</body>
</html>

Step 5: Login

OK, this page doesn’t say what exactly to do after the username/password is entered. There’s no POST action. Instead, what they do is call the saveettings() function on clicking of the login.gif image. saveettings() is in the cookie.js file:

function savesettings()
{

if (document.forms[0].rememberme.checked)
{
createCookie('nasusername',document.forms[0].username.value,2);
createCookie('type',document.forms[0].type.value,2);
createCookie('nasrememberme',1,2);

}
else{
eraseCookie('nasusername');
eraseCookie('type');
eraseCookie('nasrememberme');
}
document.forms[0].action = "newlogin.do?phone=0";
document.forms[0].method = "post";
document.forms[0].submit();
return true;
}

OK, so the page ‘newlogin.do’ is to be opened as a response to the clicking of the login button. And the username and password info has to be passed along, of course.

We already have the base url for the login page that we just used. Now we have to combine the base url with the ‘newlogin.do’ page instead of the ‘login.do’ page that we accessed earlier:

login_form_id = "newlogin.do?phone=0"
type = "2"

login_form_url = base_url.group(1) + login_form_id

login_data = urllib.urlencode({'username': username, 'password': password,
'type': type})

f = opener.open(login_form_url, login_data)

… and success! This is enough to get the login done. I added config file handling to the final code so that the username/password are stored in a config file. The final code also ensures that we’re on a Tikona network before proceeding with the steps of logging in (by checking if the redirect URL is obtained in Step 1). See the latest code here.

Step 6: Auto-login on successful connection

Just one last step remains: a NetworkManager dispatcher script that will invoke this login program each time a network becomes ready:

#!/bin/sh

if [ "$2" = "up" ]; then
/home/amit/bin/tikona-auto-login || :
fi

Put this in /etc/NetworkManager/dispatcher.d with the appropriate permissions (744) and we’re good to go!

Next steps:
The project surely isn’t complete: a lot of support has to be added to NetworkManager itself to present a good UI to enable/disable these dispatcher scripts and also to prompt for a username/password instead of storing in a config file. This and several other TODO items are listed in the README file. If you plan on adding new networks that can be auto-logged in to, it’s easy to follow these steps or feel free to email me for guidance.

8 thoughts on “Auto-login to web proxies using NetworkManager”

  1. >Its very good…

    I have one problem… I am using Linux on my desktop. How to Login Tikona from remote location?

  2. >Cool script … very helpful

    Can you update the README in the source to mention "Put this script in /etc/NetworkManager/dispatcher.d with the appropriate permissions (744)"

  3. Nice work man, works perfectly after i added the script to /etc/NetworkManager/dispatcher.d with the 744 permissions

  4. Hi,

    Very nice post. I am not a geek in python, but can you please suggest me a way to do this in Windows.

    I have a ISP who sends a cat5 cable to my home. I plugged it in my wifi router and access internet from my laptop. the problem is that if internet connection is not used for quite sometime, I will need to enter my credentials. it is a proxy with a 172.xxx private IP. it is a plain authenticating server with no scripting on it.

    I am currently using ROBOFORM to automate this but not so effective. I’ve set this up to run on logon but one has to restart the machine or logoff-logon to do this. Instead of this pain, we can simply enter the creds manually.

    can you show me something like this:

    my computer pings google.com or any internet address every 1 minute. if no response, try to run a script that will open my proxy address and automatically logon.

    because in your python script, it says connection is up or down, but wifi connection is always up in my home regardless of internet is ON or OFF.

    please help

  5. Is there any way to bypass this inventum login page and access the net directly without being redirected from inventum……..???????
    If there is even any slight chance or method is available then pls tell me….

  6. The method described appears to be a bit complex. I found a workaround using 3 freeware applications, namely cmdow.exe, autoit.exe and Take Command LE.

    These are the steps. Every thing is done through a .btm batch file run under TakeCommand LE. It is linked to start at PC startup.

    This is assuming that you have a single internet connection and that you are connected to the Tikona modem directly or through a router and both are switched on. If through a router it must be set to “Automatic IP” under wan settings. Mind you not all routers operate properly ! My Asus RT-N-!6 works perfectly while Belkin does not.

    Here are the contnts of the batch file. You would of course have to edit it for correct path.

    =================================
    @echo off
    start /wait netsh interface set interface “Realtek” Enabled
    delay 3
    start /wait netsh wlan connect name=ASUS-PMS interface=Realtek

    unset trial
    set /a trial=0

    delay 3

    :loop

    set /a trial=%trial+1

    if %trial=5 goto quit

    unset googleping

    set /a googleping=%@ping[www.google.com,5]

    delay 5

    if %googleping LE 0 goto next

    goto skip

    :next

    start /pgm “d:\Pale Moon\Palemoon-Portable.exe” 1.254.254.254

    delay 10

    d:\Scheduler\

    d:\Scheduler\cmdow.exe “Tikona Digital Networks – Pale Moon” /max

    delay 10

    start /wait “c:\Program Files (x86)\AutoIt3\AutoIt3.exe” “d:\AutoIT\MouseLocate.au3″

    delay 10

    d:\scheduler\cmdow.exe “Tikona Digital Networks – Pale Moon” /res

    delay 3

    start /wait “c:\Program Files (x86)\AutoIt3\AutoIt3.exe” “d:\AutoIT\ClosePaleMoon.au3″

    goto loop

    :skip
    :quit
    exit
    ==============================================
    What ping portion does is first checks if you are already logged in. If you are it skips the whole process and exits. You are already connected.

    If you are not connected it opens Tikona login page in “Pale-Moon Portable” browser window.

    Next cmdow command maximises the window.

    The 1st autoit script moves the mouse over the login button and clicks it once.
    The batch file waits for defines time for you to login and display the logged in page.
    Next cmdow command restores the window to the original starting size.
    The next autoit script closes the Pale-Moon browser.

    Here are the contents of the 2 autoit scripts.

    1. MouseLocate.au3
    ===========================
    #cs —————————————————————————-

    AutoIt Version: 3.3.12.0
    Author: myName

    Script Function:
    Template AutoIt script.

    #ce —————————————————————————-

    ; Script Start – Add your code below here
    MouseMove(1420, 575, 10) ; Move the mouse cursor to the x, y position of 1420, 575 and move instantly.
    MouseClick(“left”)

    ===========================================

    2. ClosePaleMoon

    =============================================
    #cs —————————————————————————-

    AutoIt Version: 3.3.12.0
    Author: myName

    Script Function:
    Template AutoIt script.

    #ce —————————————————————————-

    ; Script Start – Add your code below here
    MouseMove(1810, 50, 50) ; Move the mouse cursor to the x, y position of 1810, 50 and move instantly.
    MouseClick(“left”)

    ========================================

    I have 1920×1080 p monitor. You would of course have to edit the contents of the autoit scripts to your monitor.

    Thats it.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>