Whoa, it’s been a long time since I’ve posted here. I’ve been working on a lot of fun stuff lately; SomaSeek.com is running an updated version using PHP-PDO and much-improved javascript. I’m also working on integrating Sphinx or Solr search (Sphinx is a bit more appealing to me but seems like it should have a generator that would analyse your DB schema and rough out a configuration for itself), and working on a JSON / SOAP / XML-RPC API for it (which I *might* just end up doing in Node.js for funsies).
Anyway, here is a quick diddy in Node.js that should help you get started a bit more than the super simple examples they usually provide. Pay close attention to the scope of everything and how the callbacks are working..it’s a bit to wrap your head around at first.
KNOWN LIMITATIONS (Want to fix it? Get on bitbucket and submit your patch!!! https://bitbucket.org/ip2k/simple-nodejs-fetch-and-display-page-body )
- It will fail / barf if you request a page that redirects (like http://msn.com). This is because msn.com redirects to www.msn.com and the HTTP status code is either 301 or 302. Still working on better error trapping for that!
- It won’t get secondary page resources. Not sure how to solve that without involving sessions and/or forking, which is kind of beyond the scope of this right now. I’m not going to implement this unless there is serious interest in this…not sure why there would be; you’d be better off using a proxy.
var request = require('request'),
url = require('url'),
http = require('http');
function getPage (someUri, callback) {
request({uri: someUri}, function (error, response, body) {
console.log("Fetched " +someUri+ " OK!");
callback(body);
});
}
var server = http.createServer(function (request, response) {
requestedUri = url.parse(request.url).pathname;
requestedUri = requestedUri.substring(1);
console.log("Got request for " +requestedUri);
if (!requestedUri.match('^http')) {
console.log("requested URI is not a valid URL! Dropping request...");
response.writeHead(400, {"Content-Type": "text/html"})
response.end("Invalid url");
} else {
getPage(requestedUri, function(body) {
response.writeHead(200, {"Content-Type": "text/html"}),
response.write(body),
response.end("ip2k.com NodeJS simple server demo")
})
}
});
server.listen(8000);
console.log("Server running at http://127.0.0.1:8000/http://example.com");
console.log("To use, just append some URL as a request, like this: http://127.0.0.1:8000/http://example.com");