But what is stuff? And what is the internet? Very good questions!
Since you are able to reach this blog, which is on the internet, I'm going to assume that you have at least a super-basic knowledge of the internet. That is, that you can use web browsers to visit web sites.
In this post, I'll teach how to write web-applications. By that, I mean writing web sites that do some fairly complicated tasks. Another name for web-applications is dynamic web sites. Basically, it's a web site that interacts with web user (a.k.a. client.) This is contrasted with a web site that does not interact with the clients (a.k.a a static web site). For example, this very web-site is very minimally dynamic: you can rate and add comments at the bottom of the page. If I were to disable those, this web site would be completely static. In a dynamic web site, the client can change something (such as add a comment, etc.) on the web site. In a static site, all the web client can do is get stuff. An example of a static web site is The Best Page In The Universe, as no one can leave comments or change the content in any way. An example of dynamic web sites are Google (the page you get depends on the words you typed), Facebook (news feeds, recommendations, ads, what-not, depend on content people gave Facebook), etc. etc. etc.
There is a caveat, however, and to understand it, I'll have to go into how web sites work.
How Web Sites WorkWeb sites have two parts: the part that runs on the servers (the computers that belong to the web site), and the part that runs on the client (the browser running on your computer.)
Clearly, there is some interaction going on there. To simplify things, it's good to think of what happens as a transaction, that is, the browser asks for a web page from the web server, and the web server responds with that web page. When the transaction is over, the web server and web browser are free to go on with their business. They can effectively forget about each other, until the web browser initiates another transaction. In other words:
- Only the web browser can initiate a transaction.
- A transaction happens very quickly.
- When the transaction is finished, it is finished.
Of course, just because a transaction is finished, it doesn't mean that the browser or server won't remember anything. They may, but there is nothing "built-in" that forces them to. For example, a browser can keep a history of places you've visited. And a web site (especially a dynamic one) may save information from that transaction as well (such as your Twitter tweets.)
Let's "zoom in" on that transaction for a moment. We said that the browser initiated a transaction. This may happen by a person typing in "http://www.google.com/" in their address bar thingi. The browser goes to Google's web site, and gets something back. That something is called HTML. If you're using Chrome, and your right-click on any web site, you should see a menu option called "View page source." If you choose that, you will see the HTML that you got back.
Your trusted web-browser got that HTML, analyzed it, and made it look all nice and pretty: made it look like whatever it is you're seeing when you're browsing around. HTML can contain links to other web pages. If you click on such a link, another transaction is initiated and another HTML document is returned to your browser. HTML can also contain forms, and if you fill it out and hit "submit" or "search" or "whatever", then another transaction is initiated, the browser sends whatever it is you filled out to the web server, and the web server gives back another HTML doc. That's the way it works.
Sort of. Let's zoom in a little more. We mentioned that the web site responds to a web request with HTML. But how does it figure out which HTML to respond with? That greatly depends on the nature of the web site, and there are not hard-and-fast rules on how to do this. For example, a static web site may just be a bunch of HTML files laying around in a computer folder somewhere. A browser requests a certain file, and the server responds by giving back the contents of that file.
A more complicated, or dynamic web site will have some sort of programming running on that server that figures out what to do. That program can be written in just about any language imaginable! Python is one of these languages, and lucky for you, Python actually gives you great tools to write sophisticated web sites with relative ease. But you are not limited to Python. You can write web sites in C++, Java, and even languages that were specifically invented for writing dynamic web sites (such as PHP and ASP.)
So what is this demon I speak of? Let me put it this way. When you wrote your little Python scripts, you downloaded the Python interpreter, wrote the code, and had it run. You can have it do anything you want, you can have it completely destroy the computer if you wanted too (I leave writing a quick program that tries to erase every file on the computer as an exercise to the student.) And that's Python, a fairly advanced language.
- HTML5: that's the basic HTML, what describes the structure of any given web page. What are its links, where are its images, etc.
- CSS3: that's describes the looks of a web page. What fonts should be used, how are things spaced, the colors, etc.
- cry a lot, and realize that you can't do it. Then cry some more.
Exercise 3: check out Bootstrap's templates that make creating simple, yet beautiful, static web pages easy! They take advantage of sophisticated JS & CSS libraries, including Font-Awesome, to simply development. If you need to add some nice pictures, check out Pexels for some images free of charge. Makes you want to create a web site, doesn't it? :)
Oh No! Another Tangent!!!
In the previous tangent I discussed the transaction that takes place between a web browser and a web server. Before going into the details of the server-side in the transaction, I want to give you a little more background about how the internet freaking works!
Say you type "http://www.idocarmi.com/p/pytutorial.html" in your browser and hit go! Many things happen to get that transaction going.
First, the browser needs to find the server. How can it do that? Well, http://www.idocarmi.com/p/pytutorial.html is called a URL that describes to the browser the steps in needs to take to initiate the transaction.
In that URL, there is www.idocarmi.com, that's the domain name (or host name) of the web server that it needs to find. The browser can then, ask a domain name server (DNS) for the IP address that that domain name refers to. IP what? Every computer on the internet has a number. That number is called its IP-address. Most IP addresses are four numbers separated by a period. To see what I mean, you can go to host2ip, type in any domain name that you want (www.google.com, www.idocarmi.com, etc.) and it'll give you its IP address (something like 188.8.131.52). Every computer has a name for itself called localhost, as well as an IP address for itself: 127.0.0.1.
Once the browser figures out the IP-address, it can use cool little internet protocols to figure out how to send data to the web server. How that happens is also very interesting, but for another day (the curious can read about the OSI model.)
It may be shocking to discover that the internet is used for many purposes, and not just web browsing (sending emails is just one of many other things that can be done.) Each such purpose has an internet protocol that define how two or more computers on the internet interact with one another. For web browsing, the internet protocol is called HTTP, hence the "http://" at the start of the URL. Secure (encrypted) web browsing is governed by the HTTPS (HTTP-Secure) protocol. Luckily, HTTPS is just HTTP with encryption lapped on-top of it. So once you know HTTP, you don't need to know much more to do HTTPS, just everything that there is to know about security.
But that's not all. In the basic Python tutorial you should have learned that computers are dumb. Therefore if the client wants to talk HTTP, but the server is hearing SMTP (used for emails), as you can probably imagine, at best nothing will happen, and at worse, a big mess will happen. Most protocols aren't designed to let the computer on the other end know which protocol is being used. It's not like to do HTTP, the first thing the browser says is "I'm going to do HTTP now." It would be nice if that how things worked, but it's not. Instead, another way was devised to enable one server to talk multiple protocols at once. This is achieved using ports.
A port is just a number between 0 and 65,535. The "standard" or "reserved" ports are under 1024. For example, the standard port for HTTP is 80. The standard for HTTPS is 443. The ones for SMTP are 25 and 587. The significance of these numbers is that now a computer on the internet can be used for many purposes at the same time without confusion about what is happening at any given time. It's important to remember that these numbers are just standards and recommendations. No one is stopping you from running an HTTP server on port 8080, or port 25 for that matter. It's highly advised that you not use a port under 1024 unless it's the standard for that protocol (i.e. 80 for HTTPS). But anything over 1024 is free for you to do as you please. For instance, if you wanted to run an HTTP server on port 8888, you can do that, but in order for the web server to know that you it's running on port 8888 instead of 80, it will have to be explicitly told to do so in the URL, like this:
(Note the colon followed by 8888 at the end of the host name in the URL.)
Of course, there is nothing listening on that port for the www.idocarmi.com server, so if you type in that URL in your browser, you'll get a webpage not available message from your friendly browser.
You can also be explicit about port 80 in the URL:
But it's the same as leaving it out, as it is the standard port for HTTP. In fact, your browser may conveniently "erase it" for you.
You can also be a bastard, and so something like this:
Now you're telling your browser to talk HTTPS (secure) on port 80 (non-secure HTTP). It'll try, but it'll fail because the server will talk HTTP back. You should get an SSL connection error of some sorts. SSL is basically the technology behind the 'S' in HTTPS. :) While also very interesting, this too shall be for another day (the curious can read about Public-Key Cryptography.)
What's left in the URL is all that stuff after the host. In my example its:
This tells the browser to get the file called "/p/pytutorial.html" Or more completely, it tells the browser to use the HTTP protocol to ask the server named www.idocarmi.com and which is listening on port 80 (implied) for the file found at path "/p/pytutorial.html". QED.
(BTW, I don't want to mislead you into thinking that all internet protocols are transactional in the same way that I described in the previous section. HTTP and HTTPS are, and these are the protocols that you'll be familiar with in web development.)
So is that it? Are we FINALLY ready to get started on Python programming for the web?
Almost. I don't want to get into all the details, but there is more to the HTTP protocol than the URL that I've just described (and more to the URL than I have just described too.)
HTTP also allows for parameters (can be thought of like parameters of a Python function) to the web server, and thus allow more structured data to go through.
Another feature in HTTP is that with every response, the web server responds with a status code. For example, 200 means "all is well", while 500 means "something is seriously broken". 404 means "i can't find what you're looking for." (and thus a broken link.)
That's pretty much it for now. There are more online explanations of http and cookies. If you want the actual protocol reference, you can look up the relevant RFC (see http and cookies). But at this point (and possibly ever) these RFCs may be too much information.
"Please remind me why I'm reading this post?"
To learn to do web development [using Python.] "Ahhh, now I remember, yes!"
As I hinted previously, Python has some really cool tools to help you in your web development. Each has a different approach, with advantages and disadvantages each.
django - a very comprehensive framework. It has a lot of built-in features that help you build super-complicated stuff super-fast. The draw back is that it has a somewhat steep learning curve, and making little modifications may be complicated. You need to understand how it works and work with it, not against it. It a bunch of powerful reusable apps that you can download an "plug into" your project. It also has a built-in admin interface that lets you develop sophisticated data-entry websites quickly and easily. In a sentence: it's a wonderful world, but one that takes getting used to (for help on that, see my post on Django hints & tips.)
flask - in many ways it's the also-powerful opposite of django. It has a very low learning curve, and comes with some very powerful tools to help you with. But the simplicity comes with a price: you have the tools, but now you need to work hard to use them in order to build something useful. If you're doing something very unique, this may be your best bet. Otherwise, you may be longing for a few of those django reusable apps.
There are many more frameworks, but I believe that (at the time of this writing) these two are the dominant ones. While very different from another, they both share a few key (and wonderful) features:
ORMs (Object Relational Mappers)
This is a powerful technology that lets you work with databases (huge data stores containing complicated, and related information) without having to know anything about databases! They let you use object-oriented programming (Python classes) to add, retrieve, update, and delete information. Django has ORMs built-in, while Flask is very well integrated with sqlalchemy.
Clean & Smart Template Model
This can be very hard and messy to achieve. In the PHP & ASP programming language, the code is deeply embedded in the HTML, making trouble-shooting and testing quite difficult. With both Django & Flask, there is a clean separation between the abstract work that the web application is performing (information processing, etc.) and how it is presented to the end-user (HTML generation.) Django calls this MTV (or MVT). The Model is the basic information structure. The View describes which information is then presented (that is, given back to the web browser.) While the Template describes how this information is to be converted to HTML.
As is the theme with Django and Flask, Django has this template model built-in, while Flask is very well integrated with jinja2.
Both Django and Flask support unit-testing. They help generate "test databases" and provide tools to help you make sure your web application is working as expected.
Reverse URLs (creating links)
Building dynamic applications is, well, a dynamic process! :) As your application changes, you want the links that are generated to be good (that is, not broken.) Both Django & Flask have built-in tools to help ensure that this never happens.
Having your web application out there begs hackers to try and mess with it. Both frameworks give you the tools to prevent many common web attacks. Of course no defense is prefect, and your web application may still be vulnerable some some attacks. And although very interesting as well, this discussion will be left for another day. (The curious can look in OWASP's Top Ten.)
Both support integration with many complementary technologies. More on that in a bit.
And much, much, more....
Both frameworks offer so many other features that I'll be hard pressed to name them all. Internationalization -- allowing web sites to support multiple languages / localization. Performance optimizations. Debugging tools. And of course, great online documentation (as well as a few books.)
(See, that was in a bit!)
HTTP is a complicated protocol. And to implement it well, as well as fast, is hard. Therefore, a division of labor was established in which certain web servers do the HTTP, while application frameworks communicate with the web servers and let you, the developer, write code mostly oblivious to HTTP. Common (and free) web servers that are used in production environments are apache as well as nginx. But don't worry about it for now, as both Django & Flask have built-in web servers that are great for development purposes. And even if you'll want to go to production, you can leverage services that take care of the web servers as well as other complementary technologies for you. (A quick search got me django and flask)
Another common complementary technology is a database server. This is the technology that stores all your valuable information. It is also the technology that the ORM in these frameworks tend to shield you from very effectively. But even if you're not interacting with them directly, you still need to use them! A great free such database that is good for development purposes is sqlite. What's great about it is that it comes built-in with Python, and you don't have to do much to set it up! Unfortunately it is not so great for many production environments. In those cases, you may want to use postgresql or mysql (both are free, and work great!) Again, you don't need to be concerned about this now. This is just something to keep in mind.
These are the big ones. Are are more, niche technologies (such as redis), but their use will depend on your specific needs.
Get on already!
Well, now you should be familiar enough with the main aspects relating to web development.
Exercise 4: go through both the flask tutorial as well as the django tutorial. Going through both will give you a better feel for what they are like and how they differ.
Seriously. :) All I can do now is give you some personal recommendations and tid-bits of information.
First, what do I recommend? Flask or Django? My honest answer is that if you have the time to learn Django, it's well worth it. I say this not because I like the learning curve, but because all the re-usable apps and the admin interface as incredibly powerful tools at your disposal.