Automating Features vs. Fundamentals

I have been in a lot of meetings over the past few years where inefficiencies in a few practices and policies were being discussed.  In terms of software development, there's a lot of things that can be done to improve how the code is written, deployed, and tested.  The meetings began talking about these possible options; however, after about a 10 minute discussion in the first meeting, everyone seemed intent on designing the tools specific to their domain and not the groundwork that is common across all domains.  Now, nothing is wrong for discussing features to standardize an approach and not reinvent the wheel every time; however, I have learned that the features are domain specific (albeit some may be more common than others) and it's the recreation of the fundamental code that saps a developer's energy, focus, and motivation usually. 

Years have went by and still the same people talk about the features without giving any thought into the remedial code that is common and takes the longest usually.  Between debating the aspects of the features and evaluating tools, such turned into analysis paralysis very quickly with very few realizing it.  Soon, the feature advocates turned more into idea zealots; preaching their views while not provided any acknowledgement to other opinions.

Because of all of this talk and no action, I began to run a few tests with some of my colleagues.  There were 3 projects that shared the same core project foundation in it's design but each developer was going to create their own.  For the sake of this story, we'll say that this was going to be a "professional" Hello, World console application that had to handle industry standard command line arguments (i.e. -? and a few others).  Each person took about 5-6hrs writing the basis of their console application; setting the proper output color schemes and the console output for the help information and other fundamental requirements.  Finally it took them about 1hr to do the actual project specific logic. Now, in this example, one could say that Developer-A creates it and then passes it along to the others in a copy/paste sort of way; however, the test needed to be done not only for time purposes but for implementation purposes since all 3 application bases were not consistent.

From these examples, we refined the foundations and eventually abstracted it into a Visual Studio project template.  The next time they they had an opportunity to use the template, it saved them that much time yet again.  This template was only the fundamentals and didn't have any actual features (no consistent way to do things like emailing or FTP services or error handling).  Later versions have added these features into it (or provided a model to make it easy to do a custom implementation) but the original versions were just the fundamentals.

In the end of this experience, the idea zealots were still preaching their features without recommending any solution to their implementation and we became more efficient by automating the fundamentals.  If you're looking to improve efficiency, try to find remedial, repetitive areas in your process and see about automating them.  Start with the fundamentals and then add features to it.  There'll be changes; however, if you Develop > Refactor > and then Abstract what works, you'll be in a much better place than if you stayed in a meeting room and debated about what features should be in the initial version of the script or template.  My Rule of Thumb from experience: Version 0 of any automation script or template should only have the minimum amount of features possible and focus on the fundamentals.

How I'm Learning F# - Working with .Net Objects and Properties

Over the past few months, I've been hearing more and more about the use of functional programming concepts and also languages like Haskell and F#.  While some of the initial musings that I've read revolved around how the concepts have been around for decades and that it makes financial and scientific applications easier to read and write, I couldn't find a good reason to start learning it for my typical line-of-business application design and development job or even some of my basic hobby projects.  Nonetheless, I kept getting drawn to the concept and have decided to focus on learning it.

This post marks the fourth entry of a series that I'll be writing to discuss how I'm going about learning F#.  While I'm not saying that my method of learning is ideal and should be followed by others, I'm just reporting how I'm going about doing it.  Over the course of this series, my goal is to provide other .Net developers a(nother) resource for learning the F# language as well as apply the language into some non-financial or scientific scenarios.

 

Series Table of Contents:

1. Finding Resources
2. Writing the First Application
3. Interacting with the .Net Framework
4. Working with .Net Objects and Properties

 

Project Overview:

In this post, I'm going to interact with libraries in the .Net framework to illustrate how to you'll be able to apply F# to functionality that normally may do in another language. Unlike the last project in this series that just brushed on using some static methods of the System.IO.File class, this post will be focusing on the various steps it takes to send an email message and working with the properties of the System.Net.Mail.MailMessage class.  In this example, we'll be doing the following:

1. Instantiating a new MailMessage object.
2. Set the To, From, Body, and Subject properties of the object
3. Attach a text file to the message.
4. Send the message via a SMTP server.

Throughout the course of this project, we'll be dealing with a large amount of F# syntax as well as a couple of assemblies from the .Net Framework.  This post will hopefully be pretty straight forward in the steps outlined above.

 

Starting With The Values:

To keep this example simple, I'm going to be declaring our To Address, From Address, Subject, and SMTP Server values first thing.  We will be using these values as we begin to set the object properties of our MailMessage object shortly; however, to allow for easier visibility, I'm setting these up prior.

let ToAddress = "MyToAddress@domain.com"
let FromAddress = "MyFromAddress@domain.com"
let Subject = "F# Email Example"
let SMTPServer = "127.0.0.1"

 

Instantiating Objects in F#:

Now that we have our values established, let's open up the System.Net.Mail namespace and instantiate our own MailMessage object. To do this, we do the following snippet of code:

open System.Net.Mail

let mm = new MailMessage()
mm.To.Add(ToAddress)
mm.From <- new MailAddress(FromAddress)
mm.Subject <- Subject
mm.Body <- "Hello, F# Email"

Here we begin to see some new syntax in F#.  The syntax to instantiating a new MailMessage object is very similar to that of C# or VB.Net in that we are setting a value using the new keyword. Next we are calling the MailMessage's To collection proptery's method of Add() to add our address we assigned initially.  After that, we are assigning the MailMessage.From property to a new instance of a MailAddress Object.  Since object properties that are not read-only are mutable (or can change), we use the <- operator to assign a value to the property.  This action is repeated to assign the Subject and Body properties on the last lines of the snippet.

 

Attaching a File:

The last item we have to do with our MailMessage object is to add the attachment.  The file we're going to use is a local text file.  For simplicity sake, we'll use the relative path to the file and assign it to a value first.  Then we'll add the file as a new attachment to our MailMessage object.

let filePath = @".\commadelimitedfile.txt"
mm.Attachments.Add(new Attachment(filePath))

 

Sending the Email:

We now have a perfectly good MailMessage object sitting in memory.  The only thing that's left to do is to send it out.

let client = new SmtpClient()
client.Host <- SMTPServer
client.Send(mm)

The Full Project Source:

   1: #light

   2:  

   3: let ToAddress = "MyToAddress@domain.com"

   4: let FromAddress = "MyFromAddress@domain.com"

   5: let Subject = "F# Email Example"

   6: let SMTPServer = "127.0.0.1"

   7:  

   8: open System.Net.Mail

   9:  

  10: let mm = new MailMessage()

  11: mm.To.Add(ToAddress)

  12: mm.From <- new MailAddress(FromAddress)

  13: mm.Subject <- Subject

  14: mm.Body <- "Hello, F# Email"

  15:  

  16: let filePath = @".\CommaDelimitedFile.txt"

  17: let att = new Attachment(filePath)

  18: mm.Attachments.Add(att)

  19:  

  20: let client = new SmtpClient()

  21: client.Host <- SMTPServer

  22: client.Send(mm)

 

Summary

This has been a very short and simple example on how to interact with the .Net framework objects and their properties.  At this point, some basic and typical tasks that are used in applications could be transposed to F# if you so desired.  With F# being a concise language, it has some benefits over traditional object-oriented languages.

jQuery, JSON, and ASMX 2.0 Services

A few weeks ago, I had a project that was grounded in the .Net Framework v2.0 and Visual Studio 2005.  The requirements were very focused on usability and speed so AJAX and jQuery were high on my list to use.  Through the project, I learned that there isn't a large amount of information in one place that tells you how to setup a ASP.Net solution that uses jQuery and ASMX services to effectively transmit json data back and forth from the client.  Because of this, I'll attempt to fill this void since there are still many developers and companies out there that have not been able to upgrade to Visual Studio 2008.

In this post, I'll discuss the process of building a plain ASP.Net 2.0 web application project (not web site project) , setting up the necessary entries in the web.config file to utilize the ASP.Net 2.0 AJAX Extensions v1.0, and use jQuery at the client side of the transfers.    In addition, I'll also show to to use the JavaScriptSerializer class and how to write your own custom converter for your objects.

 

A Note About This Post:

This post is focused on Visual Studio 2005 with the ASP.Net 2.0 AJAX Extensions v1.0.  The v3.5 of the extensions that came with ASP.Net v3.5 and Visual Studio 2008 have a few changes.  While I will try to point out the differences, the core of this post is going to be focused on Visual Studio 2005 web application projects and v1.0 of the AJAX extensions.  While we all love focusing on the latest and greatest, I'm aware that there are a large number of companies and professionals out there that are locked into using the older version of the software for a number of reasons.

 

Additions to VS2005 Used in This Post:

This post will be using the following additions to VS2005.  Below are the items and links to their respective installers:

• Web Application Project
• ASP.Net 2.0 AJAX Extensions v1.0

 

Code Downloads for This Post:

• Visual Studio 2005 Example Project 

 

Creating and Configuring a New WAP:

Now that we have the required additions established, we're ready to create a new ASP.Net 2.0 Web Application Project.  I'm not going to get into the details of how to do this; however, I want to stress that I'm NOT choosing an ASP.Net AJAX Enabled Web Application.  I'm just choosing to create a new, basic ASP.Net Web Application Project.  Now that the project has been created, we need to add a few references and add some information into the Web.Config file.

In the Solution Explorer, we'll need to add a reference to the AJAX Extensions v1.0 library, System.Web.Extensions.dll.  By default, this library is located at C:\Program Files\Microsoft ASP.Net\ASP.Net 2.0 AJAX Extensions\v1.0.61025\ directory.  After adding this reference, we can update our Web.Config file by including a reference to the ScriptHandlerFactory HttpHandler using the following snippet inside of the <System.Web> config section:

   1: <remove verb="*" path="*.asmx"/>

   2: <add verb="*" 

   3:      path="*.asmx" 

   4:      validate="false" 

   5:      type="System.Web.Script.Services.ScriptHandlerFactory, 

   6:            System.Web.Extensions, 

   7:            Version=1.0.61025.0, 

   8:            Culture=neutral, 

   9:            PublicKeyToken=31bf3856ad364e35"/>

By adding the AJAX Extensions reference and updating the web.config file, we're now ready to enable our ASP.Net Web Services (ASMX) to be called from JavaScript.

 

Setting Up an ASMX Web Service:

We have our new WAP setup and configured, it's time to write a simple ASMX web service and configure it so that it will be able to return JSON.  To do this, let's add a web service to our project called JsonService.asmx and add the following code snippet to replace some of the defaults Visual Studio gives you:

   1: using System.Collections.Generic;

   2: using System.Web.Script.Services;

   3: using System.Web.Script.Serialization;

   4:  

   5: [WebService(Namespace = "http://YourNamespaceHere.com")]

   6: [WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]

   7: [ScriptService()]

   8: public class JsonService : System.Web.Services.WebService

   9: {

  10:     [WebMethod]

  11:     [ScriptMethod(ResponseFormat = ResponseFormat.Json)]

  12:     public string GetCustomer()

  13:     {

  14:         // Method Body

  15:     }

  16: }

Lines 1-3, I imported 3 additional namespaces to use in the code.  System.Collections.Generics will be used in the next section.  System.Web.Script.Services allow us to decorate the service and it's methods as Script methods for the ScriptHandlerFactory to use when making AJAX calls to and from the client.

Line 7 decorates the web service with the [ScriptService()] attribute.

Line 11 decorates the GetCustomer() web method with the [ScriptMethod()] attribute.  This attribute tells the ScriptHandlerFactory that this method is allowed to be called from an Ajax Client.  The properties inside of the attribute, ResponseFormat = ResponseFormat.Json, tells the ScriptHandlerFactory to send the response stream as a json string and not XML or Soap.  If a response to the web service that is not formatted as json, the response will be returned as XML.

At this point, we can create the body of our web method in any fashion as long as it returns a string.  If you are only passing base types, you can skip down to the Talking to the Server Using jQuery section; however, if you want to pass something a bit more complex, I recommend you continue to the next section.

 

Using the JavaScriptSerializer and a Custom Converter:

While passing base types is easy enough, it can be important to pass objects back and forth from the client.  In order to assist with this, the ASP.Net AJAX Extensions v1.0 comes with the JavaScriptSerializer object.  This object has the ability to serialize certain objects into strings representing json objects.  This sounds great and the answer to all of our problem!  Too bad it is very limiting in it's natural state.  It CAN convert Arrays of base types and (from what I can tell) any framework classes that implements IEnumerable<T>.  I haven't experimented with some of the more obscure generic collections; however, I do know that it does concern List<T> and Dictionary<S,V> just fine. 

In order to use the JavaScriptSerializer, you simply instantiate it and call its Serialize() method, passing the object that you wish to serialize.  If this is a string array or an object of type Dictionary<string,string>, it will do all of the heavy conversion for you and give you a nice little string to return to the client as shown below:

   1: // Method Body

   2: Dictionary<string, string> customerInfo = new Dictionary<string, string>();

   3: customerInfo.Add("FirstName", "John");

   4: customerInfo.Add("LastName", "Doe");

   5: customerInfo.Add("EmailAddress", "JohnDoe@Domain.Com");

   6: customerInfo.Add("PhoneNumber", "555-555-1212");

   7:  

   8: return new JavaScriptSerializer().Serialize(customerInfo);

In this code snippet, I have instantiated a new Dictionary<string,string> generic object and populated with the property information for a customer.  Lastly, I instantiate a new JavaScriptSerializer object and call it's Serialize method, passing our customer information into it.  The JavaScriptSerializer will create following string from our dictionary:

   1: {"FirstName":"John","LastName":"Doe","EmailAddress":"JohnDoe@Domain.Com","PhoneNumber":"555-555-1212"}

This is just a simple string that, technically, we could have concatenated ourselves; however, you see how it converts the name-value pairs of the Dictionary object and turns them into a json Object with properties and string values.

Seems pretty simple.  Now, let's turn our customer Dictionary into a CustomerInfo object with the same four properties.  Below is the class definition:

   1: public class CustomerInfo

   2: {

   3:     public string FirstName { get; set; }

   4:     public string LastName { get; set; }

   5:     public string EmailAddress { get; set; }

   6:     public string PhoneNumber { get; set;}

   7: }

Now, if we replace our Dictionary object with our CustomerInfo object we get code that looks similar to the following, easier to read, snippet:

   1: // Method body

   2: CustomerInfo custInfo = new CustomerInfo();

   3: custInfo.FirstName = "John";

   4: custInfo.LastName = "Doe";

   5: custInfo.EmailAddress = "JohnDoe@Domain.Com";

   6: custInfo.PhoneNumber = "555-555-1212";

   7:  

   8: return new JavaScriptSerializer().Serialize(custInfo);

Sadly, running the above code will give you the following error when you attempt to run it:

Since the JavaScriptSerializer doesn't know the definition of our CustomerInfo object, we get this Circular Reference error.  This causes us to go down one of two roads.  We can either turn the object manually back into our Dictionary object, or we can write a custom JavaScriptConverter for our CustomerInfo Class.  The nice thing about a custom converter is that once it's setup to our JavaScriptSerializer, we can then have it convert any number of CustomerInfo classes (or collections of CustomerInfo objects) we may need.

 

Writing A Custom JavaScriptConverter

Writing our own custom JavaScriptConverter is not as difficult as one may first assume.  Inside the System.Web.Script.Serialization namespace, we are provided with an abstract base class that helps us outline the definition and gets us started quickly.  In our project, let's add another class called CustomerInfoConverter.  Have the class inherit from the JavaScriptConverter class and right-click on the class name and select "Implement Abstract Class".  What you get is the following code snippet:

   1: public class CustomerInfoConverter : JavaScriptConverter

   2: {

   3:     public override object Deserialize(IDictionary<string, object> dictionary, Type type, JavaScriptSerializer serializer)

   4:     {

   5:         throw new Exception("The method or operation is not implemented.");

   6:     }

   7:  

   8:     public override IDictionary<string, object> Serialize(object obj, JavaScriptSerializer serializer)

   9:     {

  10:         throw new Exception("The method or operation is not implemented.");

  11:     }

  12:  

  13:     public override IEnumerable<Type> SupportedTypes

  14:     {

  15:         get { throw new Exception("The method or operation is not implemented."); }

  16:     }

  17: }

 

The first method the JavaScriptConverter makes us define is the Deserialize method.  This method is used by the JavaScriptSerializer to convert a json object from a client into the expected type of the web method uses as a parameter.  The JavaScriptSerializer automatically converts the json object into a Dictionary<string, object> collection.  Inside this method, you could add our mappings between the Dictionary keys to a new CustomerInfo object's properties and finally return that new instance of the CustomerInfo object.  Below is the method body:

   1: CustomerInfo cust = new CustInfo();

   2: cust.FirstName = dictionary["FirstName"].ToString();

   3: cust.LastName = dictionary["LastName"].ToString();

   4: cust.EmailAddress = dictionary["EmailAddress"].ToString();

   5: cust.PhoneNumber = dictionary["PhoneNumber"].ToString();

   6:  

   7: return cust;

 

The second method the JavaScriptConverter makes us define is the Serialize method.  This method is used during the actual serialize process.  Here, we're creating a new Dictionary<string, object> collection for our CustomerInfo class.  Since our CustomerInfo class will be boxed through the method's parameter, we'll need to properly cast it before we can start adding the values to the Dictionary object.  Below is the method body:

   1: // Cast the obj parameter

   2: CustomerInfo cust = obj as CustomerInfo;

   3:  

   4: if (cust != null)

   5: {

   6:     Dictionary<string, object> result = new Dictionary<string, object>();

   7:     result.Add("FirstName", cust.FirstName);

   8:     result.Add("LastName", cust.LastName);

   9:     result.Add("EmailAddress", cust.EmailAddress);

  10:     result.Add("PhoneNumber", cust.PhoneNumber);

  11:  

  12:     return result;

  13: }

  14:  

  15: // If the obj doesn't convert for some reason, return an empty dictionary.

  16: return new Dictionary<string, object>();

 

The last item the JavaScriptConverter makes us define is the SupportedTypes property of type IEnumerable<Type>.  This property should be a collection of types that this converter supports.  Since we are using this converter only for our CustomerInfo class, the property can be simplified to the following line of code:

   1: get { return new Type[] { typeof(CustomerInfo) }; }

 

Now that we have our converter built, we can attach it to our serializer to get our json string as shown below:

   1: CustomerInfo cust = new CustomerInfo();

   2: cust.FirstName = "John";

   3: cust.LastName = "Doe";

   4: cust.EmailAddress = "JohnDoe@Domain.com";

   5: cust.PhoneNumber = "555-555-1212";

   6:  

   7: JavaScriptSerializer jss = new JavaScriptSerializer();

   8: jss.RegisterConverters(new CustomerInfoConverter[] { new CustomerInfoConverter() });

   9: return jss.Serialize(cust);

 

Talking to the Server using jQuery:

Now that we have our server-side infrastructure setup, we can begin to write our AJAX client-side code using jQuery.  Compared to all of the code we've written, this is the easier part.  Below is the JavaScript/jQuery code that can be used to call our web service:

   1: function GetCustomerFromServer()

   2: {

   3:     $.ajax({

   4:         type: "POST",

   5:         url: "/JsonService.asmx/GetCustomer",

   6:         dataType: "json",

   7:         data: "{}",

   8:         contentType: "application/json; charset=utf-8",

   9:         success: function(msg){

  10:             var custInfo = eval("(" + msg + ")");

  11:             alert(custInfo.FirstName);

  12:         }

  13:     });

  14: }

In this client-side code, we're making a HTTP Post call to our web service by calling the web method of it directly.  We are stating that we are sending and receiving data in json format.  Despite the fact that our GetCustomer() web method does not take any parameters, we still need to send an empty json object in order to have json returned. 

Lastly, we write an anonymous method to handle the successful message returned to us.  The message is evaluated in order to be turned into a json object on the client side.  We then validate the object by echoing the FirstName property in an alert box.

One thing to remind, this is for ASP.Net 2.0 Web Services.  The response from ASP.Net 3.5 Web Services IS different in that the response message (msg) has it's content in a property only called "d".  So instead of eval("(" + msg + ")"), it would be eval("(" + msg.d + ")").

Summary:

This post has covered a large amount of steps to get a json-based web service infrastructure setup using jQuery and ASP.Net 2.0 web services.  After a lot of low level research and asking questions to people, I realized that there was not a single location for this information. Hopefully, this post will help fill that gap.