All the latest news and views from the world of Java

The Podcast

Podcast 6: Reactive

07-Aug-2017 in Podcasts

In this podcast we muse about Reactive. What is it, and what courses should VirtualPairProgrammers do? We talk about Spring's upcoming WebFlux framework. Also news on the recent Docker release, and what's going to be in the new Web Development course.

Transcript

Matt:I want to start this podcast by saying, "Hello! And welcome to Chapter..." Because that's what I've been saying all week. I'm in the middle of recording. It's nice to take a break and do the podcast.

Hi, Richard.

Richard:And I've been recording as well. I've been recording Docker in a very intense recording session. So exactly the same but we're here for podcast number six.

Matt:It is number six.

Richard:I didn't think we'd get this far.

Matt:No, it feels like a long time since we started yet it really hasn't been. They’ve been roughly once every two weeks.

Richard:Yep. Yep. We're keeping up a good schedule.

Matt:Indeed.

Richard:I think that the discipline is important on that. But we're going to be talking all things Java again and I'm thinking this packed session we'll be talking about web sockets a little bit.

Matt:A little bit. I just want to touch on web sockets.

Richard:A bit of asynchronous servlets.

Matt:So, yeah. Well, I want to say I've been working on that actually for the Java Web Development course...

Richard:And we'll be doing some talk on reactive and reactive frameworks in Java. And a little bit on Docker, I would imagine. So it should be a packed podcast. Where do we start?

Matt:Where do we start? Should we go, start with reactive. I think that's going to be the big topic for this one.

Richard:Yeah, it's something we've been asked a lot now. It's one of those things that's been on our to-do list for a long time. It's been bubbling up the list and I think it's finally reached the top now. The thing that is probably a pressing concern is that Spring Framework five is, I genuinely don't know when it's going to be released.

It was supposed to be March this year. But it's still at a milestone release. But it's going to be out soon and we can use it now. You can start playing with it now. The big feature in there is that it's going to include a reactive framework in at the core and it's going to sit nicely alongside spring MVC. Spring MVC won't change but the way that you would work in the reactive framework is very, very similar to how you work in spring MVC. So a very easy progression.

So, if we've got any viewers who've never done any reactive and don't know what it's about, then we should be helping them and I think this Spring Framework 5 support is going to be a nice, easy stepping stone into reactive. I'm really looking forward to that.

Matt:Do you have a sense of how, widely reactive programming as a concept is being used out there today?

Richard:Well, I think before we do that, I think it's worth ... I've come around to the conclusion that it's worth distinguishing what I'm going to call architectural reactive from programming in a reactive way.

Matt:Okay.

Richard:So the Spring support that's coming is going to be all about the programming and I don't have any sense of how much of that is being used right now. And there's certainly a lot of talk about it. But if we start with the architectural reactive then that's really hot. And, you know, it's lots of people are doing that. So we've sort of come into this quite cold and we haven't even said what reactive is.

Matt:No, good point!

Richard:And the starting point, everybody always does this, every talk on reactive will say: go to the reactive manifesto website, which I have in front of me right now. It's reactivemanifesto.org. And it is genuinely a very simple page. It's, you’d probably print it out on a sheet of A4 paper.

So, it's fairly simple to understand. And the reactive manifesto it does sound very pompous, doesn't it? But they're copying the Agile Manifesto that ... the thing that kick started Agile.

And always and it's not about programming, it's not about any particular language or any technique really. This is talking about how a system should operate when it's under stress, when it's under load.

So there are four cornerstones of a reactive system. I'll go through them all. And I am just reading off the site. So, forgive me here.

A Reactive system should be responsive. So the system should respond in a timely manner if at all possible. So it's quite vague, but easy to understand.

t should be resilient, so if a part of your system fails, then as best as possible the rest of the system should continue.

t should be elastic. So, it should be able to respond to growing demand.

nd then only thing that kind of is a specific technique or practise is the fourth, which is that a reactive system should be message driven, which is interesting... that's kind of different to the other three in that there is a specific mechanism for achieving those goals.

Matt:So, when we've talked about this before, ignoring the message driven part for a moment, we very much go along with that view of, well, we do this sort of thing ...

Richard:Exactly.

Matt:People just do it, but didn't give it a name.

Richard:Exactly.

Matt:Yeah.

Richard:So, if you look at some, I'm going to talk specifically to our customers right now. If you've done our microservice deployment course, then you build a fleet management system. Now, of course, it's very simple and very cut down obviously. So it fits into a course. But if you look at that, it ticks all of those boxes. It's not actually a complete system. So there's some rough edges on it in some places, where it needs improvement.

But, for example, if you kill off one of the instances and you can pretty much kill any of the instances that that system has and the system will still continue, possibly in a degraded fashion. So if you, for example, drop a bomb on the queue, which is the key component in the whole system. Then, sure, your vehicles are not going to be moving around on the screen anymore because the updates are not going to be getting through. But the website doesn't just fall over in a great big heap with a HTTP 500, which would have happened if you built that as a monolithic type application.

And elastic you can achieve elastic by, we have talked about this is those courses. You would achieve that by using your infrastructure, if you're using something like Azure or AWS. Then very easy to put in monitoring on those. So if a particular instance goes over a certain CPU limit, then you just fire up a new instance and the system will be able to handle the enhanced flow. So ...

Matt:So, we are, as are many of our customers, today building semi-reactive systems as much as ... we may not be necessarily doing the messaging thing all the time, but the third, the other three

Richard:Oh, well. Messaging is, yeah, well, I mean

Matt:We know how to do messaging. We have courses on messaging.

Richard:And we've got it built in. It's built into the fleetman system. And it ... I think the message driven is the only thing I'm a little bit ... I think they really mean more, it's more event driven.

Matt:Okay. Right.

Richard:Which is, you know, reduced coupling and all the rest of it. But it's very simple to understand and you're supposed to sign this manifesto so you got a badge of pride. Now what is not reactive frankly is our Virtual Pair Programmers website.

Matt:Okay.

Richard:I was ... we promised we were releasing the Docker the course in July. So as usual I translated that as any time up to midnight on the 31st of July.

Matt:Is that midnight ... I'm trying to think what's the furthest time.

Richard:I did put on the site, midnight(UTC)! And then I realised well we're not on UTC we're on UTC plus one. So I think it was actually an hour early.

Matt:Oh, well. Fantastic.

Richard:Just in sight.

Matt:This calls for a celebration.

Richard:But the point of that, is that I in releasing the course I exposed a bug on the site, which took the whole site down for three-four minutes. So it's first you knew of that Matt. I'm very sorry about that.

Matt:Thank you for telling me about that.

Richard:Wasn't a totally site failure. It was just that you couldn't see any courses, browse courses was giving that HTTP 500. And there's a classic example, you know, that shouldn't have happened. It was just one course and a piece of dodgy metadata inside it. It was poisoning an SQL query. And the whole site tips over with a HTTP 500. It's a disgrace. And so we couldn't ... what you can do if you are a reactive site is there's a little Flash banner that you can download from the manifesto site and you can add it to the corner of your page. It's like a ribbon. It's very cute.

There's no way that we could put that on the VPP site. But that's because it's a ... we I'm sure mentioned it on previous podcasts. It's a work in progress that we're migrating across to the microservice architecture. And as things stand at the minute, we've kind of got the monolith still stood there doing it's job. But all of the components for proper microservices are there. They're just not activated yet.

Matt:Yes.

Richard:So, we're getting there.

Matt:With any of these sort of projects it's a big task and it needs to be done in a thorough and diligent way with lots of testing. So it's never going to be a be, "let's release it next week type."

Richard:Yeah. Yeah.

Matt:Attitude with that.

Richard:Yes, we're moving that. So obviously we feel that pain and a lot of organisations are feeling that pain. Apropos of this I was using one of the UK broadcast sites to watch a TV programme the other night and in the middle of the programme, their site went down.

I'm going to say it wasn't the BBC iPlayer it was one of the others.

Matt:Right.

Richard:So, I'm not going to name them. But their site went down with a proxy error. So clearly they were hosting their site on a single instance and that single instance had gone down. And so the entire site was down for several hours. And I really felt for them because they are a company who rely on advertising. Thy have lost a day’s worth of revenue there, which is very, very painful.

So they need to be looking at reactive and signing up of this manifesto and working on their architecture I think. A lot of people are feeling this kind of pain.

Matt:So that's sort of an overview then of the architecture and, you know, for the companies out there who are just doing this and haven't given it a name, they got the opportunity to go and download that little image then.

Richard:Yeah.

Matt:In terms of ... so you said that's one element, the architecture. Well, the other is the coding element.

Richard:Yeah, so on the coding, I think it's in particular the message driven side of things. You’ve got to implement that somehow. And we've done that on the virtual pair programme for years. But there's an emerging, there are emerging improvements on how you do this at the code level.

And I'm just quickly reading, I thought on this message driven block of text, I thought they mentioned the observer pattern. But it doesn't, so I'm a bit disappointed about that. But it's basically what they were talking about. But we should be thinking about events.

And we should be designing our systems in a loosely coupled way, where instead of polling for things or doing blocking calls that might hold a thread or a network connection for several seconds at a time, which is inherently limiting your scalability because you only have a limited number of threads and connections. Instead of doing all that, we use some kind of asynchronous way of communicating between services and message-driven is one of them.

But there are other ways. And we've always had for example the observer pattern from classic gang of four.

Matt:So just ... let's just talk about the observer pattern because I'm very conscious that we right now don't have a course on design patterns.

Richard:Yeah, disgrace.

Matt:And a lot of programmers, and I'd include myself in this, for the very first probably a good 10 years of my career developing, I was never thinking in terms of design patterns and researching them and understanding them. So a lot of developers out there don't have that background on it.

Richard:Would it be fair to say, at the time ... so the classic gang of four design patterns book was published in '95 I think. And it was a massive sensation. At that time you'd have been working in the Microsoft .Net arena.

Matt:That's right. Yes.

Richard:And is it fair to say, I'll probably get pilloried for this, that ... I'm sure .Net is riddled with patterns now. But at the time it seemed that they were working in their own bubble and were not so into ...

Matt:I'm going to get my dates all wrong here, but I think dot net came out late in 90s.

Richard:Yeah. Yeah. It was later than that.

Matt:In fact I started ... I confess it now, working in Visual Basic.

Richard:I love Visual Basic! I absolutely loved Visual Basic.

Matt:And I still use it if I'm doing the odd macro in Excel. I think it's great.

Richard:I think ... it kind of died ... I want to digress for a second. I'm sorry for viewers who are only interested in Java. But I loved Visual Basic. They've sidelined it, is it…?

Matt:No. No. So, well, I haven't done anything with in terms of .Net recently, but certainly the latest version of Office still have Visual Basic for Applications as they call it,

Richard:Yeah.

Matt:So you can write pretty much full applications in Office. In fact, we ... I support a little charity that has its account, sorry. It has a membership application system, which I think is written in Pascal. Don't have access to the code. And they can output data in a CSV file, and they wanted to get that data into their Sage Accounting package.

Richard:Yeah.

Matt:So I created for them a spreadsheet with a very nice little VB, Visual Basic for Applications. VBA tool in it that let's them import CVS files, rewrite it into the right format ...

Richard:Oh, heroic.

Matt:But with some nice little UI screens, this kind of things.

Richard:Heroic. Heroic.

Matt:And it's really quite pleasant.

Richard:I get the impression VB.net has sort of become a less prime...

Matt:Well, it's funny. So I was a Visual Basic developer when .Net came out. And at that point everyone started gradually moving to C#. And I remember C# being described as it's VB- semicolon.

Richard:Yeah. Right.

Matt:Because obviously from Microsoft's point of view, they were pushing this thing about ... this was the first framework for Microsoft developers in a big way. So this was about learn the framework. It doesn't matter what language you use.

Richard:Uh-huh.

Matt:If you create an application in Visual Basic .Net or C# .Net you'd get the same performance, right? This was the message they were pushing

Richard:Yeah.

Matt:And so people moved to C# purely because it made you feel like a more professional developer.

Richard:Right. Oh.

Matt:Right.

Richard:Image is everything, isn't it?

Matt:It is. But I think Visual Basic .Net is still out, still being maintained. It's got a big following still.

Richard:Okay. Terrible digression. I'm sorry about that. But I got the impression that, I mean, things like MVC seemed to come to Microsoft a bit late. So I think it's fair that the design patterns were initially the preserve of C++ and Smalltalk. And then Java took them on. And I may be completely wrong on that. But anyway, I digress. So that may be why you were using this design patterns and probably didn't have the names of them.

Matt:Yes. Certainly. You're right. MVC came a lot later for Microsoft. Because you could have implemented it,

Richard:But you would have done it by hand.

Matt:You'd have done it by hand.

Richard:I went to a conference, I think it must have been in about 2009. It was one of those Dev Days from Stack Overflow. So Spolsky and Jeff Atwood were there. And it was a cheap day as well. It was a good day.

But one of the talks was a really exciting talk on how to do MVC in Microsoft.

Matt:Yeah.

Richard:And it was like, "What!" This is what year? We've been doing this for years and they were genuinely excited about it. I'm not having pop, I'm just trying to, I'm trying to assess the time scales of it really.

So anyway, on to design pattern. It's a disgrace we do not have a course on design patterns and just this week we had a suggestion from a customer, saying, "Where's your design patterns courses?"

Matt:Yeah.

Richard:Ouch!

Matt:It's one of those gaps in the library that we will fill. We absolutely will fill and it ... but let's just for the benefit of those listening who are not familiar. Let's just sort of give a very brief overview of the observer design patterns.

Richard:The observer, yes, was one of the gang of four classic patterns.

Matt:So, my understanding of it from what I recall is that very simply that you've got an object, which I think they call the subject. And another object, which they call the observer.

Richard:Uh-huh.

Matt:And the idea is that the observer should be able to register with a subject as I'm observing you and then whenever something changes on the subject, it notifies the observer there has been a change.

Richard:Uh-huh.

Matt:And a subject can have any number of observers. So, it's a way of doing, I suppose what we call, callbacks really.

Richard:Yep. Absolutely.

Matt:It's a way of any object coming along saying, "I'd to register to be notified when there's a change." And then when the change happens a callback function will run.

Richard:Perfect.

Matt:So...

Richard:So, you're doing all of that without doing any polling. It's all ...

Matt:Yes.

Richard:Yeah.

Matt:Absolutely. So, in terms of its implementation in core Java.

Richard:Ah, yes.

Matt:So everyone always says, well, it's a very messy thing to ... an unpleasant thing to do.

Richard:And you don't see people using ... so, yeah, it's in java.util I think. And it's been in there since Java 1 blah, 1.1. Or maybe even 1.0 I'm not sure.

Matt:So just in case people have used the classes and not quite realised it, the subject will be any object that extends, I'm pretty sure it's that way round, it extends a class called observable.

Richard:From the library.

Matt:From the library. And we'll talk about this is a second. But and then on the other side, the observer, will need to implement an interface called observer.

Richard:Yes, so that terminology is a bit odd. But on the observer side, that's fine. You implement an interface, so that forces you to put a method in, which is going to be the callback. That's going to trigger when something changes in the object that you're observing. So that's nice.

Matt:And that method is called update and it takes a couple of parameters. One of which is a reference to your, I'll call it subject.

Richard:Yeah.

Matt:So we can move forward, so that's pretty straight forward on the observer side. On the subject side, the observable ... So you're extending another class, which in one sense makes because then that's giving you a lot of the functionality already.

Richard:Yeah, so some functionality in the subjects. For instance the code to allow an observer to register themselves. You need a method for that. And that's tedious to write. It's quite basic, but it's tedious to write.

Matt:So although ... so extending this class, you have to call a method, I think it's called "notifyObservers()." So you decide, "Okay. Something's change here. I'm going to call this method and our observers will get to know."

Richard:Yeah.

Matt:Which, you know for those who can’t see ... obviously none of you can see, Richard is screwing his face up at this point.

Richard:Yeah, it's a slightly messy implementation. And it was the best, so that's exactly how it was laid out in the gang of four book. But the gang of four book was written for C++. And that's the kind of best job you could do at the time. But we're now years on 15, how many? 20, 22 years on from that. Blimey.

So we're long way on from that now. And there's much better mechanisms for doing this kind of thing and that's where we get into reactive programming frameworks, which has past us by a little bit. We've been a little bit slow at getting into this. So back in Java 8 we were just talking about this before the podcast, it was definitely remiss of us. We missed it basically.

We did a course on ... it is Java 8, isn't it?

Matt:Well, we did the Advanced Java topics course covers some of the features of Java 8 that weren't in Java fundamentals.

Richard:So we've got lamdas in there and we've got some threading. It's not specific to Java 8. But we talk about threading. And we start from the very basics of threading, stuff you would never want to do manually yourself. But it's good to know and then we build up to more sophisticated constructs.

But the end point of that should have been to talk about the new construct in Java 8, which was a completable future. Which ... I mean it's not entirely observer pattern, but it's very similar.

So with the completable future, you can say, "Here's a task that I want to be done." And rather than starting a thread and having to have a running thread and then having to poll that thread, and checking if it's finished or not. Instead I'll just start this work off and we'll get a notification. We'll get a call back when it's complete.

Matt:Right.

Richard:And it's not blocking. That's the key thing. You could do similar with a callable.

Matt:Yes.

Richard:From Java 5 and we had callable in our course.

Matt:We did.

Richard:So it's well worth watching that, if you have no idea what I'm talking about, do watch it. But the problem with a callable is it is blocking. You do have to poll it to determine when it's finished. You call a get method on it and that get method waits. And then when it gets its response back it continues.

So completable future was actually a big deal in Java 8. But it wasn't sort of heralded. It wasn't made a fuss of. And it's one of those things that is, that has become just a key tool in many developer’s arsenals. And we missed it. So, sorry about that. Our bad.

Matt:So at some point we'll do a little video I’m sure on that.

Richard:Well, it might be. So what I'm leading to with all this reactive stuff is we've got to have a reactive course. And this is going to be my next, definitely what I'm doing immediately after Docker is done. This is my work.

And so completable future could do it for you. You could use completable futures instead of potentially messages to achieve the reactive goal.

Matt:Okay.

Richard:So, the easiest way to look at it is on microservice architecture. It doesn't have to be microservices, but that's a good way of thinking about it. So we have a microservice that needs to call another microservice on another instance. That's going to be a rest call.

Matt:Yes.

Richard:Up until now, we would have implemented that as a blocking call. A regular HTTP call and that blocks. HTTP is blocking by design.

Matt:Yes.

Richard:Are you going to talk about asynchronous in a short while. But, yeah, that will block. If we couldn't stand the ... so that, a blocking call is bad generally because we're holding a network connection. We're holding a thread. It might take a minute to complete. So the user’s sat there with a spinning hourglass or whatever on the website. That's not reactive. It's not responsive.

So if that had been a problem in the past, we would have gone, "Well, okay. We'll implement that as message instead. We'll send a message to a queue." It's a bit complex. With a completable future, much better. You wrap the rest call into a future, in a future task. And we'll just get notified when it's done.

Matt:Right.

Richard:So that's kind of where this is going. So, that could do the job for you, but people have realised that we all kind of need more than that. So, there's been quite a gold rush of frameworks appearing that add to that and add features onto of that.

And I musy confess, I get completely lost when I look at all these frameworks. There's RX-Java and there's a framework called Reactor. And Spring Framework are introducing their own constructs, which I'll talk about in a minute.

But, I get really confused by all of that. But it seems to be settling down and parts of this are going to be implemented into Java 9 as standard. Basically the improvements, the completable future, will have constructs that allow you to stream.

So the problem with completable future is it just works on a single task. Here's a task, do it, tell me when it's done. With Reactive streams, instead you say to your microservice that you're calling, "I want some data from you. Tell me when you've got some data." And it will send me data as and when it becomes available.

Matt:Right.

Richard:So you really do see now this is much more like the observer pattern.

Matt:Yes.

Richard:And much more like a queue without needing a queue, it's just tell me when you've ... and you use web sockets underneath all of this so you don't have connections open all the while.

So all very nice but as I say there is lots of different frameworks implementing this and there's going to be some consolidation of that.

Matt:That last you've just said about not having connections open all the while, is for me fascinating because the stuff I'm working on is all about actually that, managing your client server connections ...

Richard:It's all tied into together, isn't it?

Matt:And everything comes together quite nicely. So it's nice the way we're talking about this on the podcast because we nearly just had this conversation.

Richard:Yeah. Yeah.

Matt:Never got to repeat it. So yeah.

Richard:Definitely. So the starting point will be, I think this is the way I'm going to do it ... I might do ... I think we should do more of this rather than doing these all singing all dancing course. And I've said this on previous podcasts, a half hour, let's kick the tyres of something. Let's get you started. A quick hello world type thing.

So in Spring 5, they ... the framework is called Webflux.

Matt:Okay.

Richard:So this is the framework that will sit alongside MCV and it's lovely because your webflux controller looks just like an MVC controller. It's just they’ve added a couple of constructs on: a mono and a flux. And all that is, is a mono is a future completable future style thing, where it's going to return a value. So that's a traditional construct.

But you also have a flux, which is going to send you a stream of values possibly a finite number of values. You’re just expecting 10 or whatever, or they can be infinite. So we constantly get values from this. And it looks like they've done a very nice job of packaging it up. So that's going to be the starting point.

Matt:Okay. Very good.

Richard:I'm in very early stages, however, and I realise that I've been waffling a bit there. And I'm still feeling my way through it to be honest and it's, yeah.

Matt:Interesting. I say, I feel like it's my turn to waffle a bit there.

Richard:Good.

Matt:So I've been, and again it does tie in actually quite nicely. So as I mentioned before, I'm currently working on the new Java Web Development course to replace our existing one.

Richard:Yep.

Matt:And as a bit of a teaser, because it does tie in quite nicely, I'm currently working on the chapters that are going to cover asynchronous web services and websockets. And there's just as a sort of lead up to that. When, and I don't know how much people will be aware of this, and this is going to be on the course as well.

But when the web first started, so HTTP 1.0, if you tried to retrieve a web page under that, each connection between the browser and the sever could do one thing only. So the process would be browser connects to server, gets the html for the webpage, and disconnects. It looks at the html as says, "Okay. there is a CSS file." Makes a connection. Gets the CSS file. Disconnects. And it has to do that for each of the images and the Javascript files.

So there were lots of connections being opened and closed.

Richard:Yes.

Matt:HTTP 1.1 comes along and says, "Well, we'll allow you to open a connection and make multiple requests and get data back for each one."

Richard:Yep.

Matt:One of the things that allowed to happened, is something that's called Comet. So Comet is the web specification that says that you can ... because you can keep a connection open for longer, you can rather than doing this request, response, request, response, do a request multiple responses and there could be a pause between each of those responses.

Richard:Yes.

Matt:And the common specification says there were two ways to implement this. One of them is called long poling and that is, if anyone out there is familiar with Ajax and we're talking about Ajax on the course. That is where you, the client requests a response from the server and waits for however long, you know, you can put a timeout on it.

But effectively waits until the server responds. And when the server responds it then sends the next request. So you're sending one request at a time. But those requests, there's always a request outstanding. Like that's the real idea.

Richard:I see. Right.

Matt:So that's method one. And that absolutely you can do in Java. It's not too difficult.

Richard:So those subsequent requests are modelling an acknowledgement. I got that data and now I want the next piece.

Matt:There's no actual acknowledgement. So what happens is that the server is sending data whenever it gets it because there's this request outstanding. I don't want to talk too much about how it works. But it uses this thing called asynchronous servelets.

And the issue here, which is why it's relevant to you is that, if your client has opened a connection on the server, when you're using a web server such as Tomcat the ... each time the client opens a connection it uses a thread. And there's a pool of these threads. And the only reason that you need an asynchronous servelets is that, that pool of thread is going to get exhausted.

It's going to block and it means no further clients can connect. So all asynchronous servelets is really doing is allowing your connection between the browser and the server to stay open on a thread outside of Tomcat's thread pool. It's giving it a thread that sits elsewhere.

So you still need one thread per connection.

Richard:Yes.

Matt:With asynchronous servelets. So this is not a, for most cases, scalable solution where you'd have thousands of clients.

Richard:Sure.

Matt:I'm using as an example, that the cases studies of a restaurant. So customers can come along and place orders for collection later at this restaurant. And I'm saying, well, maybe the kitchen, which is going to have a screen showing all of the incoming orders, that could work because you've got one of these long polling...

Richard:Yeah. Right.

Matt:Connections rather than only their customers. So, it's got limited use, but it has got use.

Richard:Well, I first came across this in a company who were doing monitoring of freezers. So this was for warehouses holding stock, frozen stock of course. And they have thermometers on them. And if thermometers start going up they need to raise an alarm. And it would literally alarm somebody and they would have to go in and switch the freezers back on or something.

And then wanted to use this for that purposes. And that sounds to me perfect. I didn't really understand it at the time and I thought, "This can't be right. This isn't scalable. You're holding a connection open." But for them, they're no dealing with thousands users hitting a website. It's a known bounded problem.

Matt:And interestingly, I mean, we haven't mentioned this, the reason to use this, rather than just standard Ajax, is the instant ... is you get a quicker response. And we've in those time critical situations the response comes to you straight away. With Ajax, you're polling. So you're maybe checking, "Has there been an update?" Every five seconds.

Richard:Got you. Yes.

Matt:So, the other quite often quoted use case for this is chat type things. So you know what it's like. You're on a website, one of those pop-up boxes comes up saying, "Can we help you? One of our agents is available." And then again it works because if you say yes, what happens generally is you're put into a queue. They have 20 agents. So they're only actually ever going to have 20 of these connections open.

So it is manageable in that respect.

Richard:Promise me this is never going to happen on the Virtual Pair Programme site.

Matt:Well, if it does you're going to be the agent at the other end.

Richard:My resignation letter will hit the table before you've even completed the sentence that we're adding that to the website.

Matt:So, we won't be doing that. But just ... I'll just finish the story, so ...

Richard:Sorry.

Matt:No, it's all right.

Richard:I know our marketing people listen to this podcast. So this is my best channel of communication to them, telling them that's a red line.

Matt:Okay. We force our marketing people by the way, if anyone's wondering to listen to this. It's part of their job description. So this one way of doing it is this long poling. And that's a straightforward. And there's a second way, which is called streaming. And this is the idea of rather than it, you poll and when you get a response, you've send another poll. This is a, let's genuinely keep a connection open and every time there's an update we send it down.

Richard:Yeah.

Matt:Now, as far as I am aware and I'm sure somebody out there has done this is Java, nobody's done it in Java.

Richard:Well, web sockets has blown that out of the window.

Matt:Exactly. So web sockets has come along. It's a replacement and that's how, if you really wanted to do it that way, that's how you'd do it.

Richard:And you're covering web sockets on the course?

Matt:I certainly am planning to. Yes

Richard:Fantastic.

Matt:So just as a very brief, web sockets is not a ... it uses TCP as the connection base. So it's actually not about the web despite it being called web sockets.

But it's actually where you hold a connection open between the client and the server. But it's not using the http protocol. So that has replaced the streaming and I think it's really that it's so hard to do the streaming before web sockets came along. And, but the reason I wanted to talk about all this, is here where my issue is:so, Java 9 is allegedly being released on the 21st on September.

Richard:Ah!

Matt:Around about eight weeks from now.

Richard:Well it is good to have a date. That we're definitely not going to have Java 9.

Matt:Absolutely. But one of the things that should be in Java nine and I confess I am talking here without huge amount of detailed knowledge because I haven't looked at this, is there is a new HTTPClient API I think it's called, which one of the things it does is makes it easier to establish the web socket connection.

So, one of the things that I am currently in the process of doing, meaning I am about to start, is actually looking at that because I need to make a decision, do we cover that in the course over, as well as the existing way the Java 8 way if you like of connecting with web sockets. Or is it going to be so much better, we actually say, "Look. Here's this chat on it. And if you're doing the course on the day released, this is ... you might need Java nine as release candidate, however."

Richard:Uh-huh.

Matt:I’m currently looking at that. To see how much of an improvement it is. Because it is one of those things that has been. It's one of those little ... I mean there's a lot of stuff in Java 9. This is one of those ones that actually could be genuinely quite useful.

Richard:Yeah.

Matt:There's other things it does as well as this web socket bit. But they particularly mention that's one of the reasons they'd be working on this.

Richard:Great.

Matt:So it's a replacement for an existing API, which I'm going to get this wrong now. It's called something like HTTPUrlClient or something like that.

Richard:Oh, yeah.

Matt:So it's a replacement for that. Because that is pretty old and outdated and doesn't meet a lot of the current web standards...

Richard:That needed replacing. You're right. It's going to be these sorts of things that make Java nine, you know, we're all lost on this jigsaw nonsense. And it's ...

Matt:Yes. That's the ... well, that's the kind of feature which would actually drive me to say, "Right. We need to be using Java 9 for this project."

Richard:Yes. Oh, yes.

Matt:That said, of course, I'm doing the Java Web Development course, 90 percent of that course is going to be: here's how it works. An underlying core Java before you start looking at a framework.

Richard:Yep.

Matt:What I don't know of is how you do things like web sockets in Spring.

Richard:Oh, superb! So we ... on the ... that brings us on to ... are you finished?

Matt:Yeah. Absolutely.

Richard:That brings us on to the Docker really.

Matt:You mentioned this so.

Richard:That was beautifully done. Yes. We've just released module one of Docker. It is a basic course. And I was promising to do that anyway. I think a lot of people might be a bit disappointed with how far I goes. Because it does kind of stop at an exciting point. But I'm very pleased with it actually.

There are a lot of tutorials that you can go to the Docker, is it Docker.com or Docker.org? I can't remember. You can go there and get a reasonable tutorial. So we had to do something different and I only realised after I finished it really that, yeah, if you're a Java developer this is the best way in to Docker because it's just Java, Java all the way through. We don't mess about with Cassandra containers and all that sort of thing that others do.

I'm very happy with it. It's ... a very early chapter on the course, before we've even learnt Docker. I get you to run a container. We haven't even really learned any Docker commands yet. It's just one short command line. And we're talking about a user's computer that's completely clean. They've had to installed Docker, which is very painful. You’ve installed Docker. You run one command line and a full application is downloaded in a single container.

And it runs, it happens to be a tomcat based application just to keep consistency with other courses and it is part of the fleetman application. It's not the complete microservice architecture. It's the front-end front of it. And I think that demonstrates really well how... what Docker does for you. It's that you don't have to fiddle around with environment variables, installing a database and doing all this stuff. It just cleans all of that up.

But the point I'm rambling towards is I've upgraded the front-end of fleetman to be more reactive. The problem with the old version, in the previous course, was you had to press control R to refresh your browser to get the new positions of the vehicles. So control R. Control R. So they weren't animated basically. And I thought, "I don't like that." So I spent quite so time upgrading it to use web sockets.

So I'm using Spring support for web sockets so now the position data is being streamed to the browser and they're automatically updated in real time. So it's ... that's really nice. And I know almost nothing about web sockets. So I'm looking forward to your course with Spring support. As always in Spring Boot. And I'm sounding like a fan boy. But they've done a lovely job in keeping it as simple as possible without making it useless.

Matt:Yes.

Richard:Really joyful thing to do. Now what I haven't done at any point is explain how that works. So I think again another half hour module on this this is how to get started with web sockets in Spring is very, very feasible.

Matt:No. I mean we talk about modules, though. And one of the things that we have just been talking about is so that Java Web Development course. I can only guess on how long it's going to be in total. But it's covered in looking at around 14 to 15 chapters. It's quite sizeable.

Richard:Wow. Yeah.

Matt:And it's whether we spread that into modules.

Richard:We should be.

Matt:And whether we do it as a single. The advantage obviously of splitting it into modules is purely that we can release module 1 while module 2 is being recorded.

Richard:Yeah. Definitely.

Matt:And there is a logical progression. So it is structured very much as, you know, "Look. Here's where it all started. Here's the basics." And then we start, we reach a point and I'm almost there actually. I think I've got security left to record.

And at that point I'm almost going to be saying everything from this point onwards, which includes things like JSP to be fair, is a enhancement over and beyond the call. You could go to the website with what we've learned today. Now here's a load of things that are going to make it easier. And we end up with frameworks on the end, you see.

Richard:Yes.

Matt:So it may be that we could if you wanted ... I mean obviously I'm saying I've almost finished with recording it obviously there's the whole editing and post production process. So that doesn't mean it will be released tomorrow. But there's a possibility of making that probably two modules and maybe even three. Maybe move what frameworks into a third module, which is where I'll also cover JSF because I've promised that.

Richard:Yeah. Okay. Well, that will definitely be a separate module. Yeah, it needs to be separate modules. We've, since we've went to the subscription business, the idea was to release modules and release them more frequently and I fell horribly into the trap. I don't know how it happened with microservice deployment.

I fell back into doing the old style massive rollercoaster course with 30 chapters and it should have been three separate modules. It would have been better for the viewers, you know.

Matt:Yes.

Richard:Just more palatable. So, I nearly fell into that trap with Docker as well. I thought that was going to be a single module. It's much better as two modules.

Matt:And we were actually just talking earlier today about the fact that our NoSQL course.

Richard:Yes.

Matt:It's really three different things. It should be three modules.

Richard:I love that course. I'm really fond of NoSQL. And that was one of the courses where for a few of the things we hadn't used. I hadn't done anything with Neo4J, for example.

And I've really enjoyed learning about it and then presenting it. And I feel that course was a flop and isn't loved by the viewers somehow, with no evidence for that.

Matt:Well, it's interesting actually because the ... and this might be why. Is that when we assess the value of the course we have a report that we run on our systems that looks at how popular course is. And the way it works that out is some reasonably complicated metrics, but one of the key inputs is how many people do it and how far through it do they get?

Richard:Uh-huh.

Matt:Now somebody who wants to learn Mogo, for example

Richard:Yeah. Absolutely.

Matt:Might well complete the whole Mogo session and that might be the majority of people.

Richard:Definitely.

Matt:Because that's a third of the course, it ranks a bit more lower down. So spacing out the modules would actually tell us ...

Richard:And it would. I mean anyone looking at our site wouldn't know that we do Neo4J. It's just buried in this incohesive course.

Matt:So that's one of the other little jobs then. It's actually going to be, certainly on that course. That needs spiting up.

Richard:Definitely.

Matt:And the other very, very big course we have is Hibernate. I don't know if that one, I haven't looked at that course recently. But I don't know if that one is also a candidate for splitting or not.

Richard:I would probably leave that. It's kind of a legacy course. It's still valuable. I mean if you're using Hibernate or JPA that's the place to go. That's, no. That one's fine as it is.

Matt:But we'll look at splitting the NoSQL. And for those of us who are customers out there with an activate subscription who haven't looked at that course ...

Richard:Oh, yeah. It's lovely.

Matt:I really, yeah. So it covers Mongo. It covers Neo4J.

Richard:And Reddis.

Matt:And Reddis.

Richard:Yeah, and Reddis is probably out of all these not that glamorous. Mongo was the one that was getting all the attention at the time. But it's actually Reddis that's the little gem in the pack there. And something else that I've committed to do. This was in conversation with a customer is that nowhere on any of our courses have we handled stateless servers.

So the state of play with, for example, I keep going back to fleetman because it's our best example. Our fleet management system we demonstrate on the course what happens if you kill the web server, start up a new instance. Everything keeps running. Well I was a little bit crafty there because you never log into that site. There's no security on that site. So you don't have a session with that site.

And so what I've sort of craftily forgot to mention is that, yeah, when you do a switch over all of your users as instantly logged off, which is obviously not acceptable. So you need a way of getting your users sessions off of our servers and to store them somewhere and the best way to do that is something like Reddis. So that again is another half hour, one hour module on how to do that. It really isn't difficult. You just put a plug into Tomcat. Very simple to do, but it's something we should have in our library.

Matt:Absolutely. And to be fair you need that. We have that on our live server because we have multiple servers providing our website. And our load balancer decides which serve you get, but of course that can change through the process of using our website. So we have that live today. It's absolutely something we should be covering. I agree.

Richard:Yeah, and very short to do. The upshot of all that is we got a lot to do. We've got a lot to be working on. So I've not mentioned Docker. Module two is well well into production. So that will be this month I think. Yeah. It's public ... promised on the site as this month. And that's going to push, so module one was about, this is how Docker works. These are the Docker commands. Nothing complicated on it. But I hope it was a fun course to work on.

In Docker module two, it's about working with multiple containers and orchestrating those containers so, you know, we're going to have many, many, many containers in a production system. How do get those those onto instances? Do you put them onto one instance or hundreds of instances? How does all of that work? And all of that will be addressed with a follow on course, which we're doing Docker swarm. Docker swarm is beautiful.

Matt:So this will also I presumably allow you to do things like if your system is getting heavily overloaded you can fire up additional ...

Richard:Absolutely all managed by Docker Swan.

Matt:Fantastic.

Richard:The one nasty thing actually is that I started developing our series on microservices quite a long time ago and took the decision that we would include the Netflix stack. Netflix stack. Very popular. And probably not used by that many organisations but is a very good way of managing a microservice architecture. Things like Docker Swarm came along after that.

And in many places have replaced what Netflix was doing. So we end up with a situation, I have ported the fleet an system to a fully Dockerized containerized, but that's why that course is well advanced. Small matter of writing or presenting it. But I've got the case study done.

And, yeah, you can take it exactly as it is and drop all the Netflix stuff onto the containers and it works. But actually you've got Eureka which is a very complicated component. That's doing the service registry. So every microservice registers with Eureka to say, "Here. I am this. This is my IP address. This is my port and this is my name."

And then users, consumers of that service will look up using Eureka. Well, you can do all of that now using Docker swarm. A container contains the service and has a name. It's a naming service. You don't necessarily ... I can't really see the purpose of Eureka anymore. So there's going to be kind of an unpleasant crunching of gears when we bring that together. And I don't know how that's going to pan out yet. Interesting.

Matt:Well, I should look forward to seeing that. Certainly.

Richard:Well I know you've been enjoying the Docker course.

Matt:You've never said a truer word. Because there's nothing I enjoy more than listening to you for hours on end. Actually I haven't watched Docker yet. I'm looking forward to watching it. But because I have been working on Java Web Development I've been reviewing some of the older stuff, which of course you did on Java web.

Richard:On years ago. Yeah. It needed doing. I'm very pleased you're doing it.

Matt:Fascinating that in the course you show screenshots of certain websites and how much some have changed and some haven't is fascinating.

Richard:Ebay I think was on ...

Matt:Ebay was on and Amazon as well. Just the general Amazon site. And do you know what? It's amazing how much they haven't changed. And when you take a step back and look at, you know, what people think of good and modern web design, and the too probably biggest most popular websites out there have not changed too much.

Richard:Amazon works.

Matt:Exactly. It works. And both of them work.

Richard:And but their back-ends… I can guarantee that Amazon is reactive.

Matt:I'm sure. Yeah. Yeah.

Richard:Have you ever known Amazon to be down for any period of time. It just doesn't happen.

Matt:No. Or Ebay…. That’s the same. It’s a very resilient site.

Richard:It didn’t use to be. Ebay was originally implemented in, I think it was Pearl scripts, I think. And they famously went to Java, they were kind of like the, Sun paid them, I think, to use J2EE. So they went down the EJB route, etc. Which I’m not sure how well that went, really. Erm, but now, I don’t know what they use now, they might still be on the Java stack, but yeah, they’re never down. Their website is atrocious, it looks horrible

Matt:And yet people don’t care about that – it works. And that’s fascinating really. I mean this isn’t one that you show, or I show, but Paypal, which is the same company as Ebay I think?

Richard:Not now, they’ve split now.

Matt:Well they constantly redesign their site, and every time it becomes harder and harder to use. And there’s one particular feature, called IPN Notifications that I can never find. It’s hidden somewhere. It’s really important.

Richard:A crucial business function, yeah.

Matt:So great design is not everything, and I just wanted to…

Richard:Yes, so we’re using this as an excuse for the Virtual Pair Programmers site!

Matt:Well as long as it’s functional and it works, which I believe it does.

Richard:I think we might change the purple, the milk tray colour scheme, I think

Matt:For those who are outside of the UK, Milk Tray is a very well known box of chocolates, which is made by Cadbury’s I think,

Richard:That must be worldwide?

Matt:I don’t know… but Cadbury’s use purple, it’s their colour, but anyway. It wasn’t done with that intention.

Richard:So, we always promise a new topic for the next podcast in 2 weeks time, should it be on design patterns? Let’s do gang of 4 design patterns. That will be fun, I think.

Matt:OK,

Richard:It should be easier than reactive anway. I’ve got to say Matt, I’m really impressed, it took 6 podcasts, before you mentioned how much good work you do for charity, I don’t know how you held back!

Matt:Well if anyone is genuinely listenting to this, I hope you’ve been entertained, and/or informed,

Richard:And yes, well we’ll see you in 2 weeks, for the next All Things Java podcast.

Matt:Richard won’t like me saying this, but as always if you have any feedback, any things you’d like us to talk about, do get in touch.

Richard:And it’ll go to Matt. I’m too busy with my charities to answer any of your calls!

Matt:Yes, well we’d better sign off so that we can get on with our philanthropic work. I will look forward to talking to you again in a couple of weeks Richard.

Richard:See you next time.

Matt:See you next time.

Listen via:


Be notified of new posts