Application Design and Web Forms

As those who have read ~Sterling TNG will know, I've recently begun revamping my ~Sterling web site. I am still working on it in short bouts when I can and have gotten the first revision of my form handling stuff put together and tested using the first revision of my database security plugin.

The two major things I've struggled with over the last few years in desiging my framework is URL handling and form handling. The essential problem is that HTTP is a cruddy protocol for designing web applications and interactive systems. HTTP has only barely hacked together support to allow state to exist between clicks. As such, you have this situation where you have to track work flows page by page by page. Instead of the more typical GUI design of event by event by event. A single page might include hundreds of events, which cannot be processed by the server until the user clicks a "Submit" button or something similar.

Anyway, I've come up with what I consider to be a solution a little different from most that I've encountered. One thing I want is the ability to have truly hidden fields. Many forms need to have extra information associated with the form. Such things can include the type of operation being performed, limitations on the input the user can give, or just extra data that needs to be associated with a form that has already been set by the user or that the user doesn't get to have a say in. This is especially needed in the case of multi-page "wizards" where each page collects a little bit more data from the user. After several pages, the wizard is completed and all the data can be processed.

What I've done is assigned every form in a page an ID that is unique to each user's session. Then, session data stored on the server can be used to store the hidden information without relying on the user to submit the form properly (i.e., hidden fields in the form itself might be maliciously modified). Then each field name has this ID associated with it so that the user could have multiple instances of the same form. Each page might have multiple forms, so I've included the ability to have multiple forms on a page where each is activated separately or through a kind of event system, one form may activate another, etc. Forms that aren't activiated aren't processed, but their information might be kept in case they need to be activated later or the user hits the "Back" button.

To me, this seems really nice. I have a nice way of storing form data associated with the user and it's all fairly seamless to the programmer as well through the use of a lightweight widget API embedded in Mason files.

However, this does have a few problems: session expiration, form data expiration/naming, and storage abuses. This system requires that each form add a few entries to the user's session. Watching a user's session is a little interesting because everytime a user opens a page with a form on it, the session expands to add the metadata for that form to their session. Thus, if a person hits reload on the login page 10 times, their session expands each time. This could lead to some consequences where a form with a lot of session data could eventually consume huge amounts of hard disk space. This is a bit unlikely because most the information is small amounts of text, but you never know.

Session expiration is a problem most sites have to watch out for. For example, when I click on the Preview button of Drupal after I finish typing, I will have preceded that click by selecting the text of this post and copying it to the clipboard. Why? Because sometimes Drupal unexpected terminates a session, which will cause all my typing to be lost because Drupal will say "ACCESS DENIED" and Firefox will forget to store the values I have already typed. Nasty.

I do not want this to be a problem. So, I have to find a way to recover a user's session. Probably this will involve setting a cookie on their machine that never expires to help track the session ID they had originally. Then, when their session expires and the get taken to the login screen, the successful login will result in an automatic copying of their old session to the new one. Then, we must take the data that they submitted when the session expired and process it as if nothing had happened. My forms processor should be able to cope with this pretty seamlessly, I just need to hook it into the security API, which hasn't yet been done. It will also require some modifications to the security API which as the security API contract is incomplete.

Finally, a problem fairly unique to my setup is form data naming/expiration. This is a little more nebulous. In some cases, forms may need to be time sensitive and need the information in them to expire. I also haven't really figured out how a completely processed form should be cleaned up or if it should. What if a user logs in to the system, and then I just clean out all record of that form and then the user hits the back button a couple times to get back to the login screen and logs in again using the original form? Error! The form no longer exists. At the same time, it's bad news to store the user's password in their session longer than absolutely necessary, which would be stored indefinitely if I didn't clean up the form metadata in their session. Therefore, I'm considering setting up some way of reinitializing a form to a fresh state if they try to go back to one that's already been cleared. (Of course, this isn't yet an issue because the implementation doesn't yet clear forms out because of this problem.)

A second aspect of this problem is naming. At this time, each form widget is named with a 5-letter identifier prepended to the name to identify which form the control belongs to. This will undoubtedly break automatic form fillers, which might not be a bad thing at times. However, these tools are useful, so I don't want to completely invalidate them. Instead, I'm probably going to create some sort of aliasing system where each form element will have it's name prepended by the same integer every time and then there will be a hidden field to alias that integer to the real form ID. In the cases where a page contains only a single form, or where forms aren't nested, the integer wouldn't be necessary, but might just be added in anyway to make the coding a little easier and make page output faster. (Sometimes, I don't know if theirs going to be a nested form until after I've started printing output to the client, so we'd probably be better of just making the assumption anyway).

I hope XForms takes off because it would make all of this much easier as it gives a much more natural client-side API for handling a lot of these details (especially, nested forms, form wizards, et. al.).