Sunday, September 18, 2011

Qt5 - Introducing QDaemonApplication

Okay, so I'm not quite done with QDaemonApplication to the point where it's even testable yet. However, I wanted to at least announce it to get feedback on the structure I am using. Questions are good, and it'll help it be more robust in the end.

Also, I'd like to note that I'm not quite use to PIMPL so if there's something I did wrong in that respect, please let me know so I can correct it. I also use a slightly different programming style; however, I tried to keep it similar to what I am finding in other areas of Qt for consistency. Please let me know if there is anything inconsistent in that respect too.

So with that said...

I originally wrote about the effort in a previous blog post (see Calling Contributors and Qt5 & a major update for QtService - QDaemonApplication), and I finally found some time to be able to work on it, still with the goal to get it in intime for Qt5.

As a user of Qt5, a programmer would simply use the QDaemonApplication class much like the presently do for the QCoreApplication or QApplication classes. Though it they will also be able to do some more things between instantiating the QDaemonApplication and calling its exec() function - check parameters, etc - potentially even fully by-passing the exec() if they chose (of course, then they won't get a daemonized application, and the main program won't run - but that can be useful in certain scenarios).

Behind QDaemonApplication is a series of APIs that provide the functionality. These APIs start off with some very basic Interface classes (QAbstractDaemon*) for the Interface (e.g. command-line, systemd, Win32 SCM, etc.), Communication between the Interface program and the daemonized program, and platform integration (e.g. Win32 SCM). This structure will allow us to easily switch between different components to do different tasks - e.g. Win32 SCM vs LaunchPad vs SysV vs systemd vs upstart - and communicate in different ways - e.g. Win32 SCM, File Pipe, Network Socket, etc.

Eventually as we add more, and support more, then the interfaces, etc will be chosen when Qt5 is built, and we'll try to keep sane defaults; however, presently I am simply trying to replicate the same level of functionality that is in the existing Qt4 QtService Add-on component.

So, if you're interested in looking at what's there, even though the code documentation is thus far pretty much non-existent, you can see it at BRM-Qt5-Service on Gitorious.

Tuesday, June 21, 2011

Qt5 & a major update for QtService - QDaemonApplication

In May I proposed that QtService be integrated natively into Qt5 [1], and I offered to spearhead that task [6] and support Windows and Linux (Embedded and X11)[6]. In June I setup a branch of the Qt5 master repository[10] for supporting this task (I'll likely need to rebase it), and am working to get up to speed on using git (I have primarily used SVN), so I expect that getting going will be a little slow in that respect. (Sadly, a bit slower than anticipated. I am working on it.)

All that said, below is the summary from the Qt5-Feedback mailing list covering naming conversations, etc; and some additional stuff as I have contemplated that. The discussion has been very good thus far, but has made keeping track of via e-mail a little hard – so I am looking for a nice Wiki home for it all. I'll put it there once I find a nice home (probably at the Qt Dev Wikis somewhere) and get a chance to repost it here.

In the mean-time, please feel free to leave comments below.

Background:

QtService is presently an add-on provided by Trolltech/Nokia through the Qt Components system [1]. However, for a variety of reasons it is desirable by myself and others that it be a native part of Qt [1,3] whether as part of Qt Core or a module provided with Qt itself [1, 9]. In either case, it needs some TLC to bring it up to date as well as some improvements. To start, the existing QtService implementation is a C++ Template-based implementation[1,3]; the end result is that this prohibits use of signals/slots internally to the QtService code [1,3], prevents the ability to do a scheduled, orderly shutdown [1], and makes it hard to work with the command-line [3,5].

It has been proposed to make a new QtService implementation that makes use of C++ Abstract Interface classes instead [1,3]. In the process of doing so the ability to derive an interactive service will be removed per encouraging best practices and that it will not work on all platforms [1,3]. The new implementation should likely use a different name - e.g. QService, QDaemon, or QDaemonService - to be more consistent with existing names of parallel functionality - e.g. QCoreApplication, QApplication [5], and should address issues in the command-line [3,5], communication between controller and service [5], and add the ability to do controller shutdowns of the service [1].

Location:

I originally called for the work to be integrated into Qt Core.[11] However, after fleshing out further details we revealed several dependencies on modules – Qt SWF, Qt Network, and others. That is not to say that Qt SFW and this may not end up in the same module, but it will at least be in a separate module. For the time being, I am calling the new module QtService with the intention that Qt SFW be able to share it (more below).

Naming:

I originally proposed to use the name QDaemonService.[11] Some thought this was too long and it was proposed to just use QService.[14] However, it was pointed out the Qt SFW already uses this QService* namespace.[20,22] So, we will use the QDaemon* namespace instead to minimize confusion in the API.

Interface:

QDaemonApplication will be a formal object like QApplication and QCoreApplication, and should set up the application environment in a similar manner. That is, the command-line options provided should be available via calling QCoreApplication::arguments(). It should also have a function to tell the program whether it is the formal service or the controller so that developers can interact in both modes - thus being able to interact with the command-line as necessary.

In keeping with the naming conversions mentioned previously [5,11] the primary interface class with be QDaemonApplication. Thus the main application will look something like the following:


    #include

    int main(int argc, char* argv[]) {
        QDaemonApplication service(argc,argv);
        ...
        return service.exec();
    }


Back-End Communications:

In the QtService component, the service code used network connections under *nix and the Win32 Service Manager API on Windows for communication, which primarily relies on some IPC and command-line stuff to communicate. I think it is very important that each platform integrate something native to do the communications. To that end, I believe Qt SFW likely provides the best method of providing that functionality, and think we should collaborate between the two to utilize the IPC portion. Windows support will still require using the Win32 Service Manager API at least on the front end, and may in the back-end too so there may be some additional options of that nature. But primarily, I think we can rely on Qt SFW for IPC functionality – to provide integration for IPC, D-Bus, Shared Memory, etc – whatever is best for the platform and do so via a configurability during compile or (even better) at run-time. Otherwise, I fear we may reinvent the wheel that another portion of Qt has already finished – so why do it twice when the functionality is already there? (Yes, I realize Qt SFW was not available in Qt4 so readily. But from what I understand it will be in Qt5.)

This functionality will be hidden by the QDaemonApplication class. Developers utilizing these classes should not have to be concerned about the back-end communications – it should just work from their perspective.

Front-End Interfaces:

In the QtService component, the service code used the command-line as the sole front-end interface. However, QDaemon* should integrate with various systems, as well as keeping that simplistic command-line interface. So, while the command-line interface will be the first-out-of-the-gate supported front end, we should also add configuration support (likely build-time only) for supporting other mechanisms – e.g. systemd, zeroconf, upstart, bonjour, launchpad, etc. Once we have moved beyond supporting solely the command-line then native mechanisms will be set to be the defaults for each platform when such functionality is available (e.g. launchpad on Mac). In the case where there may be several different mechanisms (e.g. Linux – systemd, zeroconf, command-line, D-Bus, etc.) then we may select an appropriate default – e.g. D-Bus or command-line for Linux.

The primary idea here is that since there are so many different front-ends for driving service/daemon applications, it should be configurable with appropriate defaults selected. It should be as easy as possible to enable integration of new front-end interfaces for future expansion.

This functionality will be hidden by the QDaemonApplication class.

Developer Interaction:

As the application will daemonize itself in the QDaemonApplication object, developers will need to have an interface object with that class. To this extent, Developers will be required to create a class derived from an abstract interface class – QAbstractDaemonObject – which is then registered in some manner (function/signal/slot) with the primary QDaemonApplication object.

Class Architecture:

The QDaemon* namespace will consist of two public classes:

QDaemonApplication
QAbstractDaemonObject

And a number of internal classes to provide the various mechanisms for setting up the environment, interacting with the front-end APIs, etc.

The primary purpose of QAbstractDaemonObject will be to provide sufficient interfaces for developers to utilize both pre-demonization and post-daemonization. The QDaemonApplication object will do most of the work in bringing up the application, however, it will not daemonize the application until the exec() function is called – thus providing the developer time to interact with the pre-daemonized process. By allowing the developer to derive from this interface, we can also provide sufficient means to enable communications for the developer between the pre-daemonized and post-daemonized process – for custom communications (likely serializing to and deserializing from a QByteArray) via standardized signals/slots.

Instances:

Some platforms (e.g. Windows) only allow a single instance (primarily determined by the installation location and name of the service as registered with the Win32 Service Manager API) of a service to operate at a time. Other platforms could care less. To this degree, QDaemonApplication should contain the ability to differentiate between platforms and inform the developer if it is allowed, and if it is provide the developer with an easy means (boolean option) on whether to allow it or not.[14]

I think that's enough to get some discussion going.

[1] http://lists.qt.nokia.com/pipermail/qt5-feedback/2011-May/000246.html
[2] http://lists.qt.nokia.com/pipermail/qt5-feedback/2011-May/000247.html
[3] http://lists.qt.nokia.com/pipermail/qt5-feedback/2011-May/000253.html
[4] http://lists.qt.nokia.com/pipermail/qt5-feedback/2011-May/000256.html
[5] http://lists.qt.nokia.com/pipermail/qt5-feedback/2011-May/000259.html
[6] http://lists.qt.nokia.com/pipermail/qt5-feedback/2011-May/000262.html
[7] http://lists.qt.nokia.com/pipermail/qt5-feedback/2011-May/000266.html
[8] http://lists.qt.nokia.com/pipermail/qt5-feedback/2011-May/000267.html
[9] http://lists.qt.nokia.com/pipermail/qt5-feedback/2011-May/000264.html
[10] https://gitorious.org/~benjamenmeyer/qt/brm-qt5-service
[11] http://lists.qt.nokia.com/pipermail/qt5-feedback/2011-June/000449.html
[12] http://lists.qt.nokia.com/pipermail/qt5-feedback/2011-June/000450.html
[13] http://lists.qt.nokia.com/pipermail/qt5-feedback/2011-June/000453.html
[14] http://lists.qt.nokia.com/pipermail/qt5-feedback/2011-June/000454.html
[15] http://lists.qt.nokia.com/pipermail/qt5-feedback/2011-June/000455.html
[16] http://lists.qt.nokia.com/pipermail/qt5-feedback/2011-June/000457.html
[17] http://lists.qt.nokia.com/pipermail/qt5-feedback/2011-June/000459.html
[18] http://lists.qt.nokia.com/pipermail/qt5-feedback/2011-June/000460.html
[19] http://lists.qt.nokia.com/pipermail/qt5-feedback/2011-June/000461.html
[20] http://lists.qt.nokia.com/pipermail/qt5-feedback/2011-June/000463.html
[21] http://lists.qt.nokia.com/pipermail/qt5-feedback/2011-June/000478.html
[22] http://lists.qt.nokia.com/pipermail/qt5-feedback/2011-June/000500.html
[23] http://lists.qt.nokia.com/pipermail/qt5-feedback/2011-June/000525.html

Friday, May 20, 2011

Calling contributors...

Recently Nokia announced the initial planning stages for Qt5, and looked to the community for ideas on how to improve Qt in a generally source-compatible way, meanwhile allowing extensions to be added and some things to be modified. All this, via the a mailing list - qt5 dash feedback at qt dot nokia dot com. There have been a number of great ideas that have come up - from additions to the QDateTime, time-zone support, enhancing the printing, integrating more from KDE, and lots more.

So why this blog? Well, I've been working with Qt professionally for just over 2 years now making a distributed network-based geometry measurement system for the railroad industry. The design uses a lot of service applications - which, use the QtService component add-on. Well, I'd very much like to see the QtService component become part of the Qt Core library in Qt5, but it needs a bit of love to get there.

Don't get me wrong - QtService is great. It works wonderfully, but it hasn't been updated in quite a while, and doesn't officially support Qt 4.6 or 4.7. Looking at Gitoroius, it's been put into the archives - i.e. its no longer going to be officially supported. Yet, I use this every day and I'm sure others do as well.

There are also a few other relatively minor problems with QtService:

1. It is template based. And this means that the base application from which you derive your own application can't use Qt's signals/slots to deliver the basic functionality. This is of greatest hindrance in the communications between the 'controller' portion of the application (which is provided for you) and your application. It also makes it very difficult to do things like delaying a service stop request (e.g. so you can unregister the application from a central server).

2. The command line is rather limited. That is, you get what they provide you and its very difficult (actually nearly impossible) to extend it to do other things - especially if those things are passing a parameter to your application (since there is NO signal you can send to it).

So, here I am now looking ahead to Qt5 and seeing that this nice component is not going to be supported. Meaning, I'm going to have to support it myself - and so is anyone else that wants to use it too, and I'm sure there are others out there.

Of course, since it is not a first-class citizen to the Qt Framework - and you have to explicitly pull it in and install it - then I'm sure that not everyone that really could make use of it does. So there are probably a lot more people out there that could make use of it and aren't simply because its too much work to install and use it, and you get those icky limitations that aren't very friendly to you either.

Fortunately, Qt is open source. And Nokia is moving Qt to open governance, especially with Qt5. This means that I, and everyone else, have the ability to contribute to Qt5 like never before. It also means if we can get a suitable new replacement for QtService written and on par with other parts of Qt then we stand a chance of having it become part of Qt itself - a first class citizen.

Well, time to wrap up my thoughts for this post...essentially, I have now joined gitorious (https://gitorious.org/~benjamenmeyer) and will be making a branch in the next week or so to star this work on. (Very exciting). Yes, I plan to "put my money where my mouth is" or so the old saying goes. I doubt my employer will let me do it on company time, but it'll be worth it if only so I don't have to maintain the other version in a lot less friendly and open manner. (Of course, that also means pushing my employer to use Qt5 when the time comes, which is quite a bit easier to do.)

So once I get back home, then I'll be finishing the setup of my gitorious account, and creating a branch, and possibly a team, for this effort. I very much do look forward to learning git in the process (I've been a staunch SVN user for years, but I mostly do work where centralized versioning makes sense; and community projects like this make better sense with a distributed versioning system.)

So, anyone else out there that is using QtService component, or would like to join in - please join us on the Qt5 Feedback mailing list mentioned above, and we'll get you plugged into the new work once I get it all setup.

And certainly look for more here as this endeavor continues. I'll certainly try to post more as it comes together.

Thursday, February 10, 2011

Portable software...

Over the last year and a half I've had the opportunity to work with Qt in software development, and I've enjoyed it greatly.

In a previous job, I looked at what it would take to port our software from being just on Microsoft Windows to also running on Linux, Mac, and various UNIX operating systems. In the course of that research I looked at several options: (i) Qt, (ii) Gtk, and (iii) WxWidget. The problem I ran into then was (a) Qt simply cost too much for the project (due to our budget), and was a security concern - not from the bug aspect but from the aspect that the target market was military and getting Qt certified for the environment would be hard - and a failure there could keep the project from certification as well. WxWidgets and Gtk both has the same security concerns; however, Gtk also had a licensing concern given its LGPL nature and our software was proprietary - Qt provided a great way around that if we could afford it, but we couldn't. So alas I was unable to get into Qt at that time, but as a result I was able to write a good portion of code for doing the same thing - writing my own platform abstraction API.

Now, of the three I loved the ideas and concepts introduced by Qt the best. Gtk, at least at that point, was still very much Message Mapping based from what I could tell. MFC was more than enough of that for me, and was just a pain - everything was determined at compile time and there was little flexibility. One of the great concepts that sold me on Qt was the Signals & Slots system that replaced the message mapping. WxWidgets supported both. Otherwise, all three seemed to be fairly equivalent at that time.

Now, before I go on let me state that I am not trying to convert anyone from Gtk or WxWidgets to Qt with this post. However, if you are using .NET or MFC or anything else (especially on Windows) then you need to start looking elsewhere for numerous reasons which I'll save for another post.

I had written applications in MFC and Win32 for a number of years - both GUI and services. They met the need at the time they were created but are no longer sufficient. .NET, on the other hand, does seem to be vastly updated by comparison but still has quite a few issues - at least patent wise if you want to have portable software.

Now, portable, multi-platform software is going to become ever increasingly important, and unless you are doing certain things that are very tied to a specific environment (e.g. extensions to Windows Explorer or KDE Plasma) then you can reach all your customers on all their platforms with a single code-base using the right tools. WxWidgets, Gtk, and Qt are some of these tools - and probably the best and most portable of all available. They are also Open Source. WxWidgets is completely public domain; while Gtk is solely LGPL. Qt, however, has several licenses - GPL, LGPL, and commercial licenses to choose from, so from a business perspective it makes the best sense - at least, as long as they continue the commercial license program; while from an Open Source perspective all three are really about equal in choice.

All that said, I must certainly say that the creators of Qt have done an excellent job and really gotten the platform right - one that is also continuously improving as well. Perfect just gets better.

So, now why do I say that?

Well, Qt is split into several modules, and they're working on making it more modular yet. Of course, you have to use the core (Qt-core) to use any of it, but the rest is pretty much optional - everything from networking to XML to services (daemons), and more - even in-program scripting. Additionally they also made it very easy to convert from Qt to Standard C++ and back - most all of their classes have functions to convert back and forth where overlapping occurs. The layers make sense and work; consistency abounds throughout the APIs.

So now with a single API you can reach from Linux, to Mac, to UNIX, to Windows; from Desktop to Server to embedded devices (tablets, netbooks, cell phones, specialty devices, etc.). There's not much you'll need a specialized, platform dependent code-base for - which basically comes down to Kernel-land software, and integration environments whose requirements prohibit being able to choose what API set you want to use (e.g. Windows Explorer ala TortoiseSVN). And you get all of it at native performance and looks, with bindings to most languages (e.g. Python, Java, Perl).

Businesses can certainly save themselves a lot of money by using APIs such as Qt, as well as preserve their businesses should anything happen to Microsoft or MS Windows - it's quite a gamble to put all your eggs in one basket, but yet so many software development houses do.

Fixing Health Care

Despite what the Health Care Industry, Congress, and President Obama would like you to think there really is a simple way to fix health care.

President Obama and the Democrats in Congress want you to believe that all the changes in their recently passed, much despised, and soon to be at least partially repealed bill is required to fix health care. However, it really does nothing for you - and it only makes the debts higher, extending entitlements where none are needed.

The Republicans don't have much to offer, but are at least doing right by trying to remove the bill that no one really ever wanted to start with - well, except the Democrats in Congress and Obama since it made them look like they were doing something when they really weren't.

So what's the correct fix?

  1. Insurance companies will be required to accept all properly licensed doctors. E.g. eliminate the whole "out-of-network" thing; it's really just a mess that is completely unnecessary.

  2. Insurance companies will be required to pay what the doctors charge. They must not be allowed to pay out only a portion of the payment, and no refusals to pay.

  3. Doctors will be required to charge only what is necessary - they may not inflate what is charged to the Insurance companies or to individuals.


Insurance companies need to remember what their business is - betting that people will not need the benefits they pay for. However, when people need those benefits they also need to pay out. Doctors are licensed by the American Medical Association, and as such need to be allowed to make the final call, possibly having a second opinion as well, but the AMA should lay out the rules. If people opt out of having insurance they they should have to pay the full amount themselves; but the insurance industry to not and should not depend on 100% participation to work. Simply put, people that are opting in are betting they will need while those opting out are betting they will not.

Doctors ought to be able to charge what they need to. If necessary the AMA or a Federal program can provide oversight to charges - to ensure they stay within reason (e.g. costs plus a small percentage of profit). But what is charged must be payed out in full without the doctors having to appeal or inflate prices just to get what they need to stay in business, and people shouldn't have to chose a doctor based on their insurance but based on the quality of care and services provided by the doctor. As a result, people not using insurance will be charged the same as the insurance companies - and doctor's would have no reason to discount it for them as a result.

So now we've solved the same program, far more effectively, and without intruding on State or Personal rights as granted by the U.S. Constitution.

All we have done, however, is forced Congress to break its ties with the Insurance industry, their associated PACs, etc. and actually represent the people.

Monday, January 24, 2011

The End of US Entitlements...

It's time to put all US Federal Entitlement programs on the chopping block - and schedule their removal. The question is how?

To start with, Social Security has an easy, but long term, method to remove it. Take an age - say 30 (I fall in this group) or 15 or whatever - and say "You will never receive Social Security benefits". Hold to it (e.g. the age goes up every year following the group), and then close down the program as fewer people receive it. But also require that once it is shut down that all remaining funds are directly applied to the Federal Deficit if it still exists (highly likely) or should it no longer exist get paid back to everyone in the nation via a tax refund that is equally given out to everyone. (Of course you could also be generous and say just the those below a certain income level too.)

Note: If you're wondering why I chose those methods of paying it back to the people it is because it would be very hard to ensure that everyone gets paid back what they put in. So an equitable solution must be found. Of course, it also assumes there will still be money left in the Social Security coffers, which there may not be.

So there's Social Security - gone. One program down.

But what to do with everything else? What about Medicaid? Medicare? Welfare? All these programs are highly problematic to start with in how they are currently run. So let's just go ahead and shut them all down now, but we'll replace them with a single program designed to do what is at the core of those programs - helping out the needy and the poor. We'll call this program "PrimeCare". That said, here's what PrimeCare will do:

(i) if you're out of work, it'll help you get a job.
(ii) if you can't afford food, it'll help you get enough food every day - but you won't be able to go grocery shopping for it. You'll have to go to a PrimeCare facility. Transportation, if needed, will be provided as well; or in the alternative the food and supplies will be delivered to you.
(iii) if you need medical support, then PrimeCare will provide several insurance equivalent options.
(iv) if you're having trouble paying your bills, then PrimeCare will help you through debt management and bankruptcy if necessary, possibly even temporary low interest loans if necessary.

What's the purpose? To provide for the needs of those that can't otherwise afford it.

What do you need to qualify?

Well, mostly you'll have to be poor and unable to provide for yourself and your family. You'll also be required to give up spending on certain things, like CableTV, etc - things that you do not need to survive. You'll also be required to file the normal tax documents.

What's the goal?

To help them get to the point where they can pay their own way, and to provide for those that simply cannot (e.g. elderly, severely handicapped, etc.).

How do we pay for it?

If we were to take a lesson from the Bible, then we'd take an easy 10% out of everyone's paycheck. However, you would actually end up paying less in taxes than you do now - where you take out 7% for unemployment, 2-4% for medicare/medicaide, 7% for Social Security, and more. All those things go away and instead it all gets replaced by a solid 10% for everyone. If you want to make it progressive, then it could be:
(a) 0% up to poverty level (defined solely be the Bureau of Labor and Statistics)
(b) 0.05*personal income*M where 'm' ranges of whole numbers from 1 to 4 and is determined by a simple table of income levels (defined by IRS) in:
(b.1) '1' is up to the national average income of the previous year
(b.2) '2' is up to the twice the national average income of the previous year
(b.2) '3' is up to the four the national average income of the previous year
(b.2) '4' is for everyone else

That, of course, means removing the entitlements and reinventing them as aid for those that need it, providing a way and incentive to get out of the incentives. It also means that we must be willing to let those who are not willing to participate in the program, or to continuously do what is necessary to continue receiving aid, to exit the program without receiving aid even if they are unable to provide for themselves otherwise. Why are such incentives and requirements necessary? To help deter people from abusing the programs like our current programs are.

Now for the Medicaid, Medicare replacement to really work we also have to change the whole health insurance industry - but this post is already enough, so I'll write about that another time.