≡ Menu

Using PHP to Scrape the Report Card from a Code School Profile – Part 1

I have been manually placing my Master badges from Code School onto my blog which, being a WordPress blog, runs on PHP. I don’t have the script running that shows the fraction completed on Paths that I haven’t yet mastered, but I can get the ones that I’ve completed.

The PHP script I’ve built so far is below. Due to some CSS I want to change, I haven’t implemented it yet. But I can say that it does indeed scrape the page. The jQuery required to display it in the sidebar I’ll share once I get the CSS issues worked out. This is very similar to the code I used to get the CodeEval profile, but it’s been refactored and modified for use with the Code School page.

<?php
    function getClass($classname, $htmltext)
    {
        $dom = new DOMDocument;
        $dom->loadHTML($htmltext);
        $xpath = new DOMXPath($dom);
        $results = $xpath->query("//*[@class='" . $classname . "']");
        return $results;
    }
    
    
    function buildContent($results)
    {
        $content = "";
        foreach ($results as $node) {
            $partial_content = innerHTML($node);
            $content = $content . $partial_content;
        }
        return $content;
    }
    
    
    /* this function preserves the inner content of the scraped element. 
    ** http://stackoverflow.com/questions/5349310/how-to-scrape-web-page-data-without-losing-tags
    ** So be sure to go and give that post an uptick too :)
    **/
    function innerHTML(DOMNode $node)
    {
      $doc = new DOMDocument();
      foreach ($node->childNodes as $child) {
        $doc->appendChild($doc->importNode($child, true));
      }
      return $doc->saveHTML();
    }
    
    
    $previous_value = libxml_use_internal_errors(TRUE);
    $profilename = $_GET['nick'];
    $profile_url =  'https://www.codeschool.com/users/' . $profilename . '/';
    $context = stream_context_create(array(
        'https' => array('ignore_errors' => true),
    ));
    $html = file_get_contents($profile_url, false, $context);  

    $class = 'bucket';
    $resultsBucket = getClass($class,$html);
    
    $class = 'mbl tac';
    $resultsMaster = getClass($class,$html);
    
    $class = 'pr-pathStatus';
    $resultsPath = getClass($class,$html);
    
    libxml_clear_errors();
    libxml_use_internal_errors($previous_value);
?>

<a href="<?php echo $profile_url; ?>" target="_blank">
    <div class="wrapper-scores">
        <?php
        $full_content = "";
        
        $full_content = $full_content . buildContent($resultsBucket);
        $full_content = $full_content . buildContent($resultsMaster);
        $full_content = $full_content . buildContent($resultsPath);
        
        /* changing h2 tags to h1 tags and inserting line breaks */
        $full_content = str_replace("<h2","<br /><h1",$full_content);
        $full_content = str_replace("</h2>","</h1><br />",$full_content);
        
        /* disabling the anchor tags on each badge by changing to divs */
        $full_content = str_replace("<a href","<div class",$full_content);
        $full_content = str_replace("</a>","</div>",$full_content);
        
        /* changing text on heading of Path Status */
        $full_content = str_replace("Path Status","Paths In Progress",$full_content);
        
        /* return the html */
        echo $full_content;
        ?>
    </div>
</a>

Accessing the Google Foobar Challenges on Chrome

Last night I was Googling Python lambda functions and a strange thing happened. The Google search results window broke open near the top and a single line of white text appeared on a black background asking if I wanted to take a test, next to a link to google.com/foobar. After clicking on the link, I was able to login and got a shell prompt.

For some reason, some letters could be typed, but others had no effect. Initially, I was using Chrome as my browser. On Safari, I found that I could type anything there.

Ironically, Chrome’s malware defenses affect the functionality of this page. If you want to use Chrome to do the Foobar challenges, you must uncheck the “Enable phishing and malware protection” under Advanced Settings in Chrome. Some other ad blocking or malware-related extensions may also need to be turned off if this doesn’t fix the problem.

Using CSS to Create Resizable Columns in WordPress with Thesis Theme

Given that most monitors today are not 17″ CRTs, I decided to change the default width for the content column on this blog such that more of the screen is used. I didn’t want to affect the width of the sidebar.

I am using the Social Triggers Skin on the Thesis 2 Theme; YMMV. The blog content width is changed in the Skin CSS under “.landing .container”, the right-hand sidebar is (predictably) in “.sidebar” and both inherit from the block “.container”.

To change the Skin CSS, go to the WordPress admin dashboard, and navigate to Thesis –> Skin Content –> click the Skin drop-down button and select Editor. On the resulting page, click the Home button and select Page. Now click the CSS button at the top of the page. This will display the CSS template for the Skin.

The CSS that must be changed is near the top of the template:

.container {
	width: $w_total;
	margin: 0 auto;
}
.landing .container {
	width: $w_content;	
}

If you’re new to Thesis, you may find these values strange; they are variables. On the right-hand side of the page, you will see a list of all the variables that can be used here, with the option to create more.

The default values that I was concerned with are as follows:

$w_total: 897px
$w_content: 585px
$w_sidebar: 312px

Since I did not want to change the width of the sidebar, I did not show the CSS code here where that last variable is used; I do not need to change this one, but I do need to know its name.

I wanted to make the default values in CSS be the minimum widths allowed, but otherwise to allow the content column to fill much of the screen. By clicking on the variables in the right-hand pane, dialog boxes for changing the values appear. I changed the existing variables and added new ones like so:

Existing:
$w_total: 80%
$w_content: calc(100% – $w_sidebar)

(Notice the use of the calc() CSS native method in the $w_content value!)

New:
$w_total_min: 897px
$w_content_min: 585px

After changing the value in the dialog box, click Save to save the new value. The CSS template now must be modified. The changed CSS is:

.container {
	width: $w_total;
	min-width: $w_total_min;
	margin: 0 auto;
}
.landing .container {
	width: $w_content;
	min-width: $w_content_min;
}

Nota bene: If you also want to change the width of the sidebar, just change the $w_sidebar variable value. It can be a static number, or a percent value. If the sidebar is set to a percent value, I recommend adding a min-width value for the “.sidebar” class in the CSS template as well.

Be sure to click the Save CSS button on the Skin CSS page once you are finished.

Now, the content and sidebar columns together take up 80% of the width of the window, but if the window is shrunk, the content column will not get smaller than its original default size!

Happy New Year!

I was able to finish 2014 by completing the JavaScript Path on Code School, and am back to working on the Ruby Path. Rails For Zombies Redux at Code School is a free course, so even if you don’t have a subscription, you can work through this one just to try out the format without paying for it. Have a happy and productive 2015!

Take Part in or Host an Hour of Code This Week!

This week, December 8-14, 2014 is Computer Science Education Week! Since its inception in 2013, the Hour of Code movement has been gaining steam. During just the CSE Week last year, 15 million students took part.

There have never been more free resources for learning the basics of programming. Code.org, the founders of the Hour of Code, has lots of ways that someone can take part in, or even host an Hour of Code this week.

One resource that I don’t see mentioned on any of these pages, though it is quite good for a free resource, is Codecademy. JavaScript, Ruby, Python, and PHP are just some of the skills that you can begin learning about there.

If you’ve never programmed before, but you’ve wanted to learn, there’s no time like the present to get started!

Code School will be Having a Black Friday Special!

I have found Code School to be a great resource both for those just learning to code as well as those who are experienced coders, but want to learn a new programming language. It has multiple paths: Ruby (including Ruby on Rails), JavaScript (includes jQuery and CoffeeScript), HTML/CSS, iOS, Git, and an Electives path for miscellaneous technologies such as R and Chrome DevTools.

The normal price for a subscription at Code School is $29 per month, or $290 per year. However, it appears that they intend to run a Black Friday special for yearly subscriptions. The price will be revealed this Friday. If you don’t already have a subscription, or your subscription is about to run out, this could be a good time to sign up!

Unfortunately, Code School is still lacking Python, though you can learn it at Codecademy (which is free).

Creating Compressed Files for Each Folder

I had quite a few folders that I wished to compress into ZIP files, all of which happened to exist in a single folder. In Windows, you can certainly compress multiple folders by selecting them all and then right-clicking and selecting “Send To” –> “Compressed (zipped) Folder”. However, this will put all folders into a single zipped file. What if I wanted each folder to have its own zipped file?

Using 7-Zip, I was easily able to do this.

Once 7-Zip is installed, open a Command Prompt and change to the folder in which all your to-be-compressed subfolders reside. The command below will create ZIP files, also in the main folder, from all those subfolders. The assumption here is that 7-Zip is installed in “C:\Program Files\7-Zip”; if it is elswhere change this attribute accordingly.

FOR /F "usebackq delims=?" %i IN (`DIR /B /A:D`) DO "C:\Program Files\7-Zip\7z.exe" a "%i.zip" "%i"

For some other options you can use, this post was of great help.