00:00:01:Ready?
00:00:05:PowerDev Meeting #6, part 2. Reggae and http.stream.
00:00:21:Silence please, or I walk through these computers lying on the stage!
00:00:24:Do not spread the Chaos!
00:00:28:Chaos with "^"? [some IRC-related joke]
00:00:31:With shoes.
00:00:33:Welcome all present here to PowerDev Meeting #6.
00:00:38:Let's start so-called official part of the event.
00:00:44:Where are the flowers?
00:00:49:This official part consists of a few presentations,
00:00:55:performed by me, and other organizers.
00:00:59:We will show some interesting things related to MorphOS.
00:01:03:Something for programmers for the start:
00:01:10:On writing networked applications for MorphOS, having no clue about network.
00:01:16:Knowledge of sockets, protocols and similar ugly things is not needed at all.
00:01:23:It can be done using a class from Reggae multimedia framework.
00:01:29:I have to get to my computer now unfortunately,
00:01:36:to be able to present you some things which will appear on the screen.
00:01:40:Then I'm leaving the stage and will control the world from a backseat.
00:01:49:Indeed, only PPC, only 32 bits, only PiS, only MOS.
00:01:55:Looks like an advertisement slogan!
00:01:58:Ok, let's go serious now.
00:02:02:MorphOS has a library of classes for media processing named "Reggae", designed by me.
00:02:09:Reggae can be used for sound, image and - in the future - video processing.
00:02:16:One of its components is "http.stream" class.
00:02:22:This component is used for streaming multimedia from the network,
00:02:28:using HTTP protocol.
00:02:32:We can stream almost everything we can download with a web bowser.
00:02:38:I will show two applications. In the first example http.stream will be acting as a data source
00:02:44:for the rest of framework.
00:02:47:We will see a program streaming audio from the Internet
00:02:51:and playing it back in realtime.
00:02:55:In the second example http.stream will be used standalone
00:03:02:to download any data available via HTTP protocol.
00:03:09:Then, these data may be used in any way.
00:03:13:Let's start from the source code
00:03:20:of a program streaming audio file from the Net.
00:03:38:I will scroll out include files.
00:03:40:Yes, this is the complete code of our application.
00:03:43:There is nothing more.
00:03:45:So in 44 lines of code we have implemented
00:03:50:network streaming, decoding,
00:03:58:and audio playback.
00:04:03:The audio is encoded in IMA ADPCM format,
00:04:12:to match available Reggae decoders with available network bandwidth.
00:04:20:It is not very high quality.
00:04:23:Bandwidth required is 21 kB/s (168 kbps).
00:04:27:Eralier tests have shown that the network here should manage such a load.
00:04:34:Before I explain the code, let's check if it works at all.
00:04:40:I want to prove you in some way, that the stream is really downloaded from the Internet.
00:04:54:That is why I've logged into a SHH account on a server hosting the file.
00:04:58:And now...
00:05:01:We are watching Apache access log live.
00:05:08:You will see that I really get the stream from there, not from some pendrive in my pocket.
00:05:18:Before you ask - you can see that some network activity is happening still.
00:05:27:We see it really live and it is a public WWW server, hosting many pages.
00:05:35:Pages containing what?
00:05:37:Educational mostly. ["erotic" - malicious voices from the back]
00:05:40:It is an university server, so it contains things like lectures, laboratory scripts and so on.
00:05:48:"The science of sex."
00:05:52:Let's just run the application.
00:05:58:The music will be...
00:06:00:Capslock key seems to be on.
00:06:02:It is not disco-polo, don't be affraid.
00:06:05:We are so dissapointed, Krashan.
00:06:09:It will be a single promoting a new CD of one of my favourite bands.
00:06:13:"Boys"?
00:06:17:"Bialystok is my city"
00:06:30:Who will guess first?
00:06:36:No logs.
00:06:38:I will turn the volume down a bit. The music will play in the background.
00:06:43:As you can see, there is nothing in the log yet.
00:06:46:This is not because I try to fool you. It is because Apache generates log entry after downloading is finished.
00:06:53:I will break it now.
00:06:59:It has been logged right now.
00:07:02:See 87.246... entry.
00:07:06:It's scrolling away...
00:07:10:This entry.
00:07:14:It is exactly the one.
00:07:20:It's gone. The server is used intensively, as you can see.
00:07:27:"You haven't yet seen intensively used server..."
00:07:33:"Enterning exec.pl is enough"
00:07:35:Anyway, look at the line I'm pointing at with the mouse.
00:07:40:A "User-Agent" request field is logged here. My application "introduces itself" to the server.
00:07:45:It is my example application, evidently.
00:07:49:"http.stream" as well and operating system version.
00:07:55:"Arrester plus characteristic."
00:07:58:Yes, some interesting papers are available on the server.
00:08:02:Except of this...
00:08:04:...we can see...
00:08:08:...maybe later.
00:08:10:Anyway, that's how it works.
00:08:19:Let the music play, we can go back to the code.
00:08:32:As you can see...
00:08:37:...the code is as simple as possible.
00:08:41:There is no typical network code at all.
00:08:47:A few words about what's happening here. I open "multimedia.class", the main Reggae class.
00:08:54:It is the master class, which controls all the rest.
00:08:59:As I send the output to the speakers, I need an "audio.output" instance.
00:09:07:I open the class here, just like an ordinary library.
00:09:12:This inconspicuous function does all the magic.
00:09:17:Makes a network connection
00:09:20:according to the tags passed:
00:09:24:URL of the wanted file,
00:09:28:type - a sound, so other media types would be rejected.
00:09:35:Additional tag with program name being sent to the WWW server. I will say more about it later.
00:09:43:I create audio.output object here, it is responsible of playback.
00:09:48:Then I just connect these two objects using MediaConnect().
00:09:56:I call a method here...
00:10:03:This method causes CTRL-C signal to be sent to my application
00:10:10:when the playback is finished.
00:10:14:The signal makes the application to exit.
00:10:17:MMM_Play() method  starts the playback.
00:10:21:Then my program just Wait()-s for the end. Streaming and playback are asynchronous to the main thread.
00:10:35:Reggae does all the dirty network job, HTTP protocol handling (with all the nasty details).
00:10:45:Reggae also recognizes file format, decodes it and plays.
00:10:49:I think the code is rather short considering the task being done.
00:11:00:Let's close it for now.
00:11:05:I have a question.
00:11:07:If the playback is asynchronous, can we send some signals or messages to control it?
00:11:21:I propose to leave questions at the end, so the movie is not too long.
00:11:36:I will answer all questions of course.
00:11:39:This part of presentation has shown Reggae possibilities.
00:11:43:I guess programmers are interested not in audio streaming, but...
00:11:49:downloading their custom data from the network.
00:11:54:An application update for example.
00:11:58:Or other stuff I cannot even imagine now.
00:12:04:As one can send some data with a HTTP request,
00:12:08:the class can be used, for example, to submit game highscores.
00:12:15:Players can submit their highscores to a server maintaining top ten.
00:12:25:I have prepared a second example.
00:12:30:The source code first...
00:12:33:This example just fetches some HTML page.
00:12:42:Then the code is simply printed to a console window.
00:12:45:As you can see this code is even shorter than the previous.
00:12:51:The main difference is we do not even need to open multimedia.class.
00:12:56:It is just not needed this time.
00:12:59:That is because we are interested just in raw data. We do not want Reggae to decode it in any way.
00:13:08:We open http.stream class only.
00:13:14:An instance of http.stream is created with required tags.
00:13:19:The name of the class.
00:13:21:The stream name is just URL to the data.
00:13:24:Only the host name here, as I just want the root webpage.
00:13:29:It may be of course full path to a resource with all possible additions.
00:13:36:MMA_Http_UserAgent tag for application identification on the WWW server.
00:13:45:And an additional tag, which I'll explain later.
00:13:51:Then, well, I pull the data from the object in a loop, and send it to the standard output.
00:14:07:Received HTML code is just dumped into a CLI window.
00:14:10:Of course received data may be processed in any desired way.
00:14:13:They may be stored in memory, saved on disk...
00:14:17:Data may contain anything, it does not matter to http.stream.
00:14:24:But does it work? Sure it does.
00:14:32:If someone does not believe me, he can go to http://teleinfo.pb.edu.pl...
00:14:38:...and display the page source.
00:14:40:There are some strange characters visible, this is because the page is encoded in UTF-8.
00:14:45:My program does not perform any transcoding, just dumps buffer to the console.
00:14:55:Additionally...
00:15:00:Let's go back to the downloading loop in the code.
00:15:07:There is no room for simplifying it further, I guess.
00:15:10:The code shows clearly, that no advanced network programming knowledge is needed.
00:15:16:It is just Amiga-like API and BOOPSI component system basics.
00:15:26:Reggae objects are just plain BOOPSI ones.
00:15:30:They are manipulated with ordinary BOOPSI methods
00:15:32:like OM_NEW, OM_SET, OM_GET and so on.
00:15:41:Reggae has also a nice feature for programmers:
00:15:48:A built-in debugging facility and MediaLogger tool.
00:15:52:MediaLogger is a Reggae event logger and viewer.
00:15:55:I will run the example again.
00:16:02:You can see...
00:16:06:...a detailed event log.
00:16:11:We have full monitoring, it shows host and TCP port used,
00:16:14:it also shows the HTTP request sent to a server.
00:16:21:Connection information here.
00:16:24:Request sending.
00:16:26:Here we have the HTTP response header returned by the server.
00:16:32:Then we can see data pulling in 1 kB chunks.
00:16:37:End of data here, the red colour suggests error.
00:16:42:Of course end of data is not really an error.
00:16:50:Object dispose at the end.
00:16:52:So, debugging an application using http.stream class is easy,
00:16:59:as every connection detail is shown in the log.
00:17:03:On the other hand MediaLogger is available to any user.
00:17:10:Programmer may want to hide some communication details of his application.
00:17:18:Assume I do not want to reveal the details.
00:17:23:This very detailed level of logging has been enforced with a tag
00:17:30:named MMA_Http_ExtendedLog.
00:17:32:It causes all the connection details (like HTTP headers) to be sent to MediaLogger.
00:17:41:Its default value is FALSE however.
00:17:43:If it is ommited, amount of displayed information is significantly reduced.
00:17:52:But, don't forget that a clever user can always use some TCP stack snooping utility.
00:18:03:For example NetworkSnoop by Marcin Kielesinski.
00:18:05:It traces all the TCP/IP stack activity, so a determined user will get these details anyway.
00:18:14:By removing the tag we can reduce log verbosity in MediaLogger at least.
00:18:25:I think now is the time for http.stream advantages and disadvantages summary.
00:18:33:It has its drawbacks and limitations.
00:18:40:It is not an all purpose component.
00:18:43:For example using it as a network layer for a WWW browser is not a clever idea.
00:18:51:Let's start from limitations, there are a few.
00:18:57:The first one is lack of POST request support.
00:19:03:It is used in things like HTML forms or file uploads.
00:19:08:POST may be handled in the future, I don't say "no".
00:19:15:I may be convinced by users to add POST support.
00:19:21:How much will it cost?
00:19:24:It will cost nothing for MorphOS.
00:19:27:But I mean convincing you...
00:19:33:The word "convincing" does not imply any currency flow ;-).
00:19:41:"But he does not deny that such a flow may take place..."
00:19:47:Let's talk freely after presentations.
00:19:52:The second disadvantage of the class is no support for persistent connections.
00:20:00:A client supporting it may create a single TCP connection
00:20:05:and download multiple files through it.
00:20:09:Webbrowsers use it extensively, as they download images just after fetching HTML code.
00:20:19:Http.stream is designed for streaming single resources.
00:20:25:That is why persistent connections are not supported.
00:20:32:The connection is always closed after data are downloaded.
00:20:38:The third disadvantage - no support for HTTPS (encrypted connections).
00:20:45:Time for advantages.
00:20:48:The first, obvious one: the class creates easy to use abstraction layer over socket API and HTTP protocol.
00:20:58:To fetch data from network, one just creates an object, pulls data and has them ready in a buffer.
00:21:08:The class is a HTTP/1.1 compatible client (conforms to RFC 2616).
00:21:18:It handles many HTTP options.
00:21:25:Let's have a look in the documentation.
00:21:32:The list of class attributes.
00:21:37:Of course I do not plan to discuss them one by one.
00:21:41:Anyone interested in using the class would read it thoroughly.
00:21:46:It is worth noting, a full documentation is an advantage too, especially in MorphOS ;-).
00:21:55:Some MorphOS libraries have vestigal docs...
00:22:02:Coming back to the list of http.stream features...
00:22:08:Of course chunked transfer handling.
00:22:13:It is a HTTP flavour used when a resource is generated dynamically, usually with a script.
00:22:23:Chunked transfer is very common and its support is obligatory for HTTP/1.1 clients.
00:22:31:Http.stream handles it then.
00:22:35:Automatic redirection.
00:22:39:It is something we aren't aware of usually, while browsing webpages.
00:22:45:Some example, I guess everyone here knows http://www.morphzone.org.
00:22:51:If we load it to a browser, it goes to www.morphzone.org/modules/news automatically.
00:22:59:A browser has been just redirected by a server. Http.stream handles it automatically too.
00:23:07:If it receives 301, 302 or 308 response code,
00:23:13:it reconnects with the redirection URL.
00:23:20:This feature can be disabled however if an application wants to handle redirections.
00:23:28:The next feature: application can specify additional request header fields.
00:23:35:It allows for authorization support, passing additional data, etc.
00:23:44:The class has also response header parser.
00:23:51:After parsing we can get separately response code, parsed fields as key-value pairs,
00:24:00:Fields are processed according to RFC 822.
00:24:05:There is an attribute for a total number of fields.
00:24:08:MMM_Http_GetHeaderEntry() method can be used to retrieve a value of specified header field.
00:24:21:The header is searched for the field and value is returned as a pointer to a string.
00:24:27:More features...
00:24:30:Two tags for controlling the network process while it's working.
00:24:40:We can get either a pointer to the process or its name.
00:24:47:Application is then able to break the network process with CTRL-C signal.
00:24:56:So if for example network process is waiting for data, or there is a network problem,
00:25:04:main application can break it.
00:25:08:Next, built-in WWW proxy support.
00:25:11:Proxy server and port can be specified.
00:25:18:Spoofing possibility, http.stream can pretend to be any browser.
00:25:25:The first attribute, UserAgent, is "nice",
00:25:29:Passed string is extended with http.stream and OS name/version.
00:25:39:The second one allows for complete User-Agent override. It is called a "browser spoofing".
00:25:48:We can make server "think" we are a Firefox, Opera or anything else.
00:26:01:There are standard attributes, like stream length (not alvays available in case of HTTP).
00:26:12:Well, that's all in this small http.stream features review.