iDO | Real Time Monitoring for the French Socialist Party Primary

October 11th, 2011 by Florian

Real-Time monitoring solution computing the online popularity of the candidates at the Socialist Party Primary 2011.

See it in action at: http://primaireps.appspot.com/

iDO | Gmelius, Towards a Cleaner Gmail™

October 11th, 2011 by Florian

Gmelius is a poly-browser extension that removes ads in Gmail™ and enhances Gmail’s UI. Try it now in Google Chrome!

iDO | Official Release of Google+ Widget

October 11th, 2011 by Florian

Google+ Widget is written in JavaScript and jQuery, using the JavaScript Object Notation (JSON) data-interchange format, the Yahoo! Query Language (YQL) and the brand new Google+ API.

Google+ Widget is served from Google App Engine, Google’s infrastructure, to ensure good performance and reliability of service.

You can try it now at http://gplusapi.appspot.com/

iDO | ScrAPI, A Mozilla Addons’ Reviews Scraper

September 8th, 2011 by Florian
<?php
/*
Author: Florian Bersier (github.com/xpressyoo)

Copyright (c) 2011 Florian Bersier, http://florianbersier.com/

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/

$errors=0;
$base = $_SERVER['HTTP_REFERER'];
$ip = $_SERVER['REMOTE_ADDR'];
$browser = $_SERVER['HTTP_USER_AGENT'];
$language = $_SERVER['HTTP_ACCEPT_LANGUAGE'];
$date = date('j/m/Y');
$time = date('G:i');
$etime = date('B');

if(isset($_POST['submit']))
{

//GET URL
$url = "https://addons.mozilla.org/en-US/firefox/addon/".$_POST['url'];

//REVIEWS AND PAGES
$file_string = file_get_contents($url.'/reviews/');
preg_match('#<b>(.*?)</b>#i', $file_string, $nb);
$nbreviews = str_replace(",", "",$nb[1]);
$pages0 = round(($nbreviews/20));
$pages1 = round(($nbreviews/20),1);
$pages = $pages0 - $pages1;

if ($pages >= 0){$pages = $pages0;}
else{$pages = $pages0 + 1;}

//USERS
$file_string2 = file_get_contents($url);
preg_match('#<b>(.*?)</b>#i', $file_string2, $users);
$nbusers = str_replace(",", "",$users[1]);

//RATIO
$ratio = $nbreviews/$nbusers;

//INITIALIZATION
$add = 0;

//LOOP REVIEWS SCRAPING

for ($i = 1; $i <= $pages; $i++) {
$oldSetting = libxml_use_internal_errors( true );
libxml_clear_errors(); 

$html = new DOMDocument();
$html->loadHtmlFile($url.'/reviews/?page='.$i); 

$xpath = new DOMXPath( $html );
$links = $xpath->query("//div[contains(@class, 'review')and
               not(contains(@class,'reply'))]"); //Do not include the comments written by the addon's developer

$return = array();

foreach ( $links as $item ) {
	$newDom = new DOMDocument;
	$newDom->appendChild($newDom->importNode($item,true));

	$xpath = new DOMXPath( $newDom );
	$review = str_replace("\"","",trim($xpath->query("//p[@class='review-body']")->item(0)->nodeValue));
	//$review = "\"".$review."\",";
	$return[] = array($review,);
} 

// REVIEWS ARRAY
$return = print_r($return,true);
$return = htmlspecialchars($return);
$return = str_replace("[0]", "", $return);
$return = str_replace("Array", "", $return);
$return = str_replace("(", "", $return);
$return = str_replace(")", "", $return);
$return = str_replace("=", "", $return);
$return = str_replace("&gt;", "", $return);
$return = str_replace("[", "", $return);
$return = str_replace("]", "", $return);
$vowels = array("1", "2", "3", "4", "5", "6", "7", "8", "9", "0", " ");
$return = str_replace($vowels, "", $return);
$strlen = strlen($return);
$add += $strlen;

libxml_clear_errors();
libxml_use_internal_errors( $oldSetting );
}

//DISPLAY RESULTS

$final = round($add/$nbreviews,6);

echo "<h4>".$url. "</h4><ul><li>Number of Reviews<br />Number of Pages<br />Number of Users<br />Ratio Reviews/Users<br />Total number of strings<br />Avg number of string per review</li><li>". $nbreviews."<br />". $pages . "<br />".$nbusers."<br />".$ratio."<br />". $add . "<br />". $final . "</li></ul><br /><p>Data retrieved on ".$date." at ".$time." from ".$browser."</p>";

}

?>

Persona – An Email Spoofing Script

September 8th, 2011 by Florian

Email Spoofing (for educational purpose only)

To be short, email spoofing is the fact to send emails using the identity of another individual or company (e.g. spam/phishing).

For example, let’s say that you want to send a fake email to one of your friends, using the identity of Barack Obama or Nicolas Sarkozy. Just use a PHP mailer such as http://emkei.cz/ and the trick is done!

Technically, email spoofing is pretty easy to achieve and just rests on the modifications of headers within the mail function of a PHP form.

Here is “Persona” a simple but effective spoofing script I wrote (for educational purpose only)

<!--?php /* Author: Florian Bersier (github.com/xpressyoo) ! For Educational Purpose Only =--> Capabilities of the code below have been intentionally limited !

This work is licensed under the Creative Commons Attribution 2.0 UK: England & Wales License. To view a copy of this license, visit http://creativecommons.org/licenses/by/2.0/uk/ or send a letter to Creative Commons, 444 Castro Street, Suite 900, Mountain View, California, 94041, USA.
*/

// Get posted data into local variables
$EmailFrom = Trim(stripslashes($_POST['EmailFrom'])); // Your email, e.g. florian@example.com
$EmailTo = Trim(stripslashes($_POST['EmailTo'])); // Recipient, e.g. email of your friend
$FakeEmail = Trim(stripslashes($_POST['FakeEmail'])); // Fake email, e.g. sarkozy@elysee.fr
$FakeDomain = Trim(stripslashes($_POST['domain'])); // elysee.fr
$Name = Trim(stripslashes($_POST['Name'])); // Your name
$FakeName = Trim(stripslashes($_POST['FakeName'])); // Your fake name, e.g. Nicolas Sarkozy

$Subject = Trim(stripslashes($_POST['Subject'])); // Subject of the email
$Message = nl2br(Trim(stripslashes($_POST['Message']))); // Body of the email

// Modify headers
$FakeSender = "X-Sender: $FakeDomain";
$FakeReturn = "Return-Path: $EmailFrom";
$Fake = "From: $FakeName ";
$Reply = "Reply-To: $EmailFrom";
$BCC = "Bcc: $EmailFrom";
$additional = "-f $FakeEmail"; \\ Hide the Mailed-by or Via
$headers = 'MIME-Version: 1.0' . "\r\n";
$headers .= 'Content-type: text/html; charset=iso-8859-1' . "\r\n";
$headers .= $FakeSender. "\r\n";
$headers .= "X-Priority: 3\r\n"; \\ Normal priority, urgent is often categorized as spam
$headers .= $Fake . "\r\n";
$headers .= $Reply . "\r\n";
$headers .= $BCC . "\r\n";
$headers .= $FakeReturn . "\r\n";

// Send Email
mail($EmailTo, $Subject, $Body, $headers, $additional);
?>

Now, to check whether an email you received is a spam or a spoofing joke/attack, always open the email in its original version/format. For instance, in Gmail, just go the top/right of your email, close to the “reply” button, click on the arrow and then on “show original”. The important fields to check are the “Received” ones (see screenshot below)

Questions/Comments welcome

Convert Web Pages and Blog Posts into PDF – Tutorial

April 19th, 2011 by Florian

Depending on the type of content you publish, it can be interesting to leave the possibility to your readers to view and save your pages or posts in pdf format.

This article explains how it is possible to achieve such functionalities by using the API of feed2pdf and WordPress.

Save a post in PDF using the Feed2PDF API

To give the possibility to your visitors to save your posts in PDF, you will need to create a simple link using the following lines of PHP:

<?php
// Retrieve the URL of the post
$permalink = get_permalink( $id );

// Create the link - the URL can be divided into 2 parts:
// 1) Feed2PDF API
// 2) URL of the post "urlencoded"
echo '<a class="pdf" href="http://api.joliprint.com/api/rest/url/print/s/feed2pdf?url=', urlencode($permalink) , '">';
?>
Download PDF</a>

Basically, this snippet (i) creates a variable called “permalink” that retrieves the current URL of your post/page and (ii) creates a HTML link that targets the PDF document generated by feed2pdf for your url. To note that the URL is encoded using “urlencode” (see PHP manual).

Read a post in PDF using Google QuickView and the Feed2PDF API

If you want to give the possibility to your readers to view your pages or posts in PDF without downloading them, then it seems interesting to use Google QuickView. Here is how you can do this:

<?php
// Retrieve the URL of the post
$permalink = get_permalink( $id ); 

// Create the link - the URL can be divided into 3 parts:
// 1) Google QuickView
// 2) Feed2PDF API
// 3) URL of the post "urlencoded"
echo '<a href="http://docs.google.com/viewer?url=http://api.joliprint.com/api/rest/url/print/s/feed2pdf?url=', urlencode($permalink) , '" target="_blank">'; 

?>
Read the article in PDF</a>

The steps are almost the same ones than before but now we open the generated PDF document through Google QuickView.

Non WordPress users

Only the first step of the process is different. Here we cannot make use of the WordPress function “get_permalink( $id )” and then need to replace the permalink variable as follows:

 <?php
// Retrieve the URL of the post/page
$permalink = "http://" . $_SERVER['HTTP_HOST']  . $_SERVER['REQUEST_URI'];

(...)
?>

Hope you liked this brief how-to. Do not hesitate to post your comments/ideas and to share this article!

Cooperation and Trust in E-Research – Identification of a Pareto-Digital Paradox by Network Externalities

April 11th, 2011 by Florian

One of the major challenges faced by electronic Research (e-Research hereafter) is the lack of collaboration or cooperation will among researchers. Up to now, the literature dealing with the topic has mainly explained the latter observation by emphasizing (i) the importance of physical or face-to-face communications to create trust and then collaboration among individuals (ii) the heterogeneity in ICT adoptions/skills among researchers and (iii) the prominence of an (old) academic `culture’ which tends to only consider or reward single accomplishments.

This short piece of work aims at proposing another perspective on the issue by adopting an economic approach. Indeed, we argue that researchers invest in relationships/links with their peers by taking into account the externalities that could result from such links.

To do so, we develop a model, in which players (i.e., researchers) agree (bilaterally) to create links in a first period (t=0) and have to make investment decisions regarding the maintenance of those links during the following periods (t >= 1). The benefits to collaborate are parametrized by the exchange of some non-rival information hold and earned by players, whereas the costs to maintain a research collaboration are captured by the play of a standard Prisoner’s Dilemma game and then depend on the cooperative behaviour of both players.

After characterization of the efficient outcomes and equilibria of the game, we identify the existence of a paradox between the Pareto efficiency requirements and the equilibrium existence conditions implied by the architecture and features of electronic or digital networks. Indeed, we show that the only social efficient outcome existing for non-empty networks that induces collaborative behaviours is not sustainable (i.e., is not an equilibrium) in large networks such as the ones created by the Internet, where many researchers may be linked to many others (i.e., asymmetric architecture of the network).

This paper proposes then to explain the lack of collaboration potentially observed in e-Research by the mismatch existing between the requirements needed to have cooperation as a social efficient outcome and the necessary structural conditions to fulfill in order to be in equilibrium.

Education in Rural India: Perspective from a North Indian Village

April 11th, 2011 by Florian

Today, 72.2% of the Indian population lives in rural areas, that is, around 750 millions of people with respect to the 2001 Census. Although it is obvious that the Millennium Development Goals regarding education won’t be achieved in overall India and more especially in the rural segments of this country, we note since more than two decades a strong and increasing willingness of the Indian government to improve the access to its schooling system, especially to its elementary education. This is achieved by a constantly growing mobilization of resources and the succession of important and original initiatives such as the Mid-Day meal Scheme, for instance.

However, Indian society and even more rural Indian society are still heavily characterized by specific features such as castes, religion, gender asymmetries, household organization, and so on. The first aim of this paper is then to put in perspective the positive changes that are expected to have occurred in the rural-Indian primary education with respect to the main socio-cultural characteristics governing these rural communities; this over a 25 years period, by taking three ‘snapshots’ of the schooling situation in a North Indian village at three different points in time, 1983, 1993 and 2008. In a second time and following a large and growing literature which tends to show that government education policies are not the primary determinants of enrollment, this paper is interested in explaining the schooling decision as a household decision, taking again into account the main features that make a rural Indian household.

Towards Better Policy and Practice Using Real-Time Data

April 11th, 2011 by Florian

Can Search Engines (SE) queries and Tweets help predict economic activity? Can Social Networking Services (SNS) help understand time use and help predict online activity?

Clearly, those two questions address different issues, but both emphasize the use of (near) real-time data to refine and improve the comprehension of our Society. Indeed, in a fast changing world such as the one we live in, it appears more and more crucial for policy and practice to be aware of what is going on in the very present in view of proposing responsive and relevant solutions in the near future. One very innovative and promising way to achieve this is by using online real-time data.

Real-time data denotes information that is delivered immediately after collection, meaning there is no delay or extremely small delay in the time-line of information provided. For example, thanks to the Twitter API , it is nowadays possible to easily gather data on what people are doing/thinking right now. Besides, tools like Google Trends provide a very simple way to collect hourly, daily or weekly data on the volume of queries related to various keywords.

Now, those tools enable researchers to get a fantastic amount of data on an extremely large set of topics. For instance, an economist could use SE queries related to a specific industry (e.g. cars) in a specific part of the world (e.g. France) to refine his seasonal auto-regressive model for this industry in this country. Similarly, it could be interesting to analyze SE queries and tweets making reference to ‘job interviews’ or to online job platforms such as ‘monster.com’ to help predict global and local unemployment rates. Finally, data gathering on the individuals’ online activity, that is, for example, the number of tweets and re-tweets per units of time (e.g. minutes) or the number of Facebook status updates per units of time could help understand when people are the most active online and, performed together with a contents’ semantic analysis, could help understand for what purpose.

This short paper aims at proposing methods to enhance current policy and practice. The first section is interested in enhancing current forecasting methods for macroeconomic activity by taking into account real-time data. The second section proposes and discusses methods using real time data to examine the time use of individuals or groups of individuals. Finally, conclusion takes place in the last section.

Email Obfuscation to Avoid Spam

October 13th, 2010 by Florian

When displaying e-mail addresses on a website, web developers have to find effective solutions to obfuscate those addresses in order to prevent spam.

Several ways to deal with such an issue exist out there, mainly CSS or Javascript driven. Whereas CSS obfuscation seems to be the most effective (see Figure – Credits: Silvan Mühlemann), the latter method has the big disadvantage to not leave the possibility to make the email address clickable, i.e. “mailto:jsmith@example.com”.

Taking into account this observation, a smart and effective way to obfuscate emails while keeping the “mailto:” feature is to use the ROT-13 algorithm in conjunction with a few lines of JavaScript.

How to do that?

This is really straightforward. First, we need to encode the email address we want to insert by using the ROT13-algorithm, which is an encoding method that rotates all alphabetic characters by 13. Then, we will decode what we’ve done using a few lines of JavaScript.

Encoding the Email

To encode your email address, you can use my online ROT13 encoder.

Decoding the Email

The next and last step is to decode and display the ROT-13 encoded email address by using those few lines of JavaScript:

<script type="text/javascript">

document.write("<n uers=\"znvygb:wfzvgu@rknzcyr.pbz\" ery=\"absbyybj\">Pbagnpg zr</n>".replace(/[a-zA-Z]/g,
function(c){return String.fromCharCode((c< ="Z"?90:122)>=(c=c.charCodeAt(0)+13)?c:c-26);}));

</script>

You just need to replace “wfzvgu@rknzcyr.pbz” by your encoded email address and copy the lines above where you want the link to appear in your page. Your visitors will read Contact me. If you want to replace the default text “Contact me” by something else, just replace “Pbagnpg zr” in the script above by any ROT-13 encoded-text you wish.

Hope you like it! Feel free to tweet/comment/share!