Maybe it’s time to revisit .NET documentation

I saw an article today that raises the possibility, for me, that .NET documentation might nearly have returned to it’s state in the early days of .NET 1.  (In those days a tool called NDoc gave great code documentation in a variety of formats.)

It’s called Taming Sandcastle: A .NET Programmer’s Guide to Documenting Your Code at Redgate’s Simple Talk blog.

This entry is a note to myself to take a look at the article when I make the time.

Note on background:

In the NDoc days I used to produce the sort of documentation that I think is right. Decent text descriptions, diagrams of the application and theory… Basically my design notes in a nice CHM, which wasn’t just a machine generated file. It wasn’t hard.  And I really love that CHM format.  It has a nice precompiled index.  Astonishingly powerful.  (I wonder if internal politics sabotaged this technology which had such great potential.)

Then along came the NDoc collapse. Even though I didn’t like the license I volunteered to lend a hand. I never even got a reply. Go figure!

I have never really dabbled in Sandcastle, having seen enough comments, from programmers who seem to share my views, that indicated it had never really arrived!

This article gives me some hope that it might be usable now.

 

Side note:  As a side project, the article above was converted to ePub format.  (This is one of the formats for eBooks.)  This apparently took a surprising amount of effort.  I tried to test the ePub file, but failed.  It could not be read by the two ePub readers that I used (MobiPocket and ICE).  Though two other ePub’s that were tested on Mobi worked as expected.  This, and the really odd designs I encountered, confirmed that this is a dynamic and dangerous field.  It also shows that you can still find programs that seem to come from an alternate universe.  (If this field takes off there will be a need for tests tools like Litmus which checks out your email in different mail readers.)

Resolving the two HTML 5 specifications

For those who find the two versions (WHATWG and W3C) of the HTML 5 specification a source of more heat than light.

On 25th or 26th June 2010, depending on your time zone, Ian Hickson, who edits both versions said in an eMail

The WHATWG doesn’t actually work on HTML5, it works on an unversioned specification for HTML that is to be continually maintained. “HTML.next”, if you will (though the spec’s title is still “HTML5″ by request from advocates…

In other words the WHATWG (Web Hypertext Application Technology Working Group) is also defining whatever comes next.

Having been distracted myself by the sometimes aggressive discussion this is an important point, that can help interested parties to better untangle what is being said.

Facebook Security

Facebook has grown from the thirst to meet up with old acquaintances, but it’s recently been strongly criticised.  I started using it with my eyes open.  I knew that security was flaky.  When the last set of security changes came through, I saw a surprising number of technically savvy users destroying their Facebook accounts.  Made me think again.

My first reaction was to also commit Web 2 Suicide (i.e. destroy my Facebook account).  Mainly because I couldn’t form a good enough opinion of what on earth was going on.  The permissions and security interface is hard to navigate, appears to be deliberately confusing and I simply didn’t understand it.

I persevered and eventually got a basic map of it.  Armed with that I realised that it would be possible to live with Facebook, provided I kept alert and put in extra work.

To assist others going down that same road, I’ve included that map and some brief notes.  This isn’t literature or journalism, it’s technical notes.

Update:  2010-06-01.  People can get hold of your information via the legal system.  I’ve called it Subpoena Leaks.  A judge needs to be convinced that a fishing expedition is a good idea.

Disclaimer

I think Facebook is a good and necessary experiment.   We don’t know what works and what doesn’t in this evolving Internet world.  Zuckerberg has retained control over it, in part by not offering you any paid services.  This enables him to do things which we (as users) can’t even imagine, the sort of things that can bring real improvement into the world.  He’s using that and in the process helping a lot of people better understand what does and doesn’t work.  I applaud the guts it takes to do that. However his attitude to Facebook users makes me concerned (profanity beware).

(Today, I hear that Facebook has acknowledged that their interface is bad and has undertaken to fix it.   Spurred on by an Attorney General or two.   There’s a chance that the interface will improve.)

Diagram

There’s a lot of ways information can get out of Facebook.  Some of them are sketched below.

FacebookDataUsers3

 

Here’s some notes on the diagram  (this is not  a description of the user interface)

The diagram shows your data in the centre.  Name, picture, profile, friend list…  It is accessed by a program layer (in grey) that feeds that information to several places.  Friends, advertisers…

Acquaintances.  Acquaintances are what Facebook calls friends.  This is the most obvious of the privacy areas, but there’s a few more as you can see from the number of coloured sectors above.  There’s a wealth of tools to manage these.  Some are hard to find but they’re there.  You can limit visibility down to individuals or the groups you yourself define.  (You can also exclude individuals or groups.)  It’s quite precise if you want.  The Individual and Group settings (at this writing) are hidden away under customize, where you also find the setting to keep that particular information to yourself.

If you want an idea of what anybody can capture from some Facebook data with a little effort, look at Pete Warden’s post on the subject.

Acquaintances of Acquaintances (AoA).  A lot of things can get published to this group, if you choose.  It’s a good idea to get an appreciation of how many they are and what you know about them.  If each of your acquaintances has between 10 and 300 acquaintances, who you haven’t friended, that could be a lot.  It could pan out between 100 and 90 000 people, that are not in your own friend list.

Everyone (on the web) Controlled like the above.  The amount of information published to everybody is increasing, without any option to limit some of it.  The recent changes to profile information (interests, books, films, music, employers and educational places) now makes that information visible to all.  (In addition to forcing you to link publicly, the automatic translation doesn’t work well and introduces it’s own cans of worms.)

Search Engines You can control what Google, Bing and the rest see.  There are web sites that automatically harvest information from Facebook and publish it online, you need to be aware what that means.  (Today I saw at a page that publishes phone numbers from lost phone number discussions, in real time.  I imagine some of those are meant to be hidden.)

Applications (Apps) are an interesting one.  I don’t know how many people use them, I personally tend to keep them disabled until needed.  Traditionally Apps have been able to download just about everything that you have stored on Facebook.  Formerly they were only allowed to hold onto the data for 24 hours before dumping it.  It’s reported that many Apps just stored the data indefinitely.  (From a practical point of view that makes a lot of sense.)  The new rule from Facebook is that App operators can hold onto data (no longer limited to a day!).  That’s a lot of data that they get.  Some of which might even be hidden from your Friends.

There is a new programming interface (May 2010) and new features that enable Apps to easily get the parts of the data that they really need.  From June 2010 I imagine that the promiscuous availability of your data will progressively diminish.  The safest way to limit applications is to deny them access altogether, if you want some access there’s finer grained controls.

There’s a Facebook Application called Privacy Mirror which lets you see how much information programmers and their applications can see.  If you use it I suggest disabling it after each time you use it.  (Try it and you’ll see why I say that!)

Application of Acquaintances (AA) is data that gets out via your Friends using an application.  Not something I expected to see but it’s there! Other applications can get your name, picture and public biography just because you’re on a friend list, but this goes further.  This could be thousands or tens of thousands of people (you’ve never met) sharing information about you with application companies, just because they’re Friends of Friends.  You can control this in that user interface.

Instant Giveaway / Instant Personalisation and  AIG / Acquaintances Instant Personalisation was recently introduced.  It’s a way for selected partners to get your information on Facebook immediately you land on their site.  At launch there are three such partners.  One of them only operates in the United States.  I find this idea creepy and want out of it, until I’ve seen how it pans out.  It’s easy to turn off your own giveaway, but acquaintances giving away your details, through this route, is harder.  You need to turn off each individual service that uses this by banning it.  That’s harsh to these services, but it’s the only option offered.

Advertisements (Ads), if I guess right, don’t get information through the same API that others use to get your data.  They are picked by a Facebook advertising engine that has direct access to everything.  So if you hide your age, they can still target adverts at your age group.  This is part of how Facebook makes money and keeps going, so it’s understandable.  There are opportunities to dig out hidden information, if the advertiser wants, so it’s potentially open to abuse.   (Facebook has been giving your ID to advertisers for some time.  From this they can get your name, picture and other details, in milliseconds.)  Currently you have no other control than to have nothing there in the first place.

Subpoena Leaks, another way all your information can be released is by court action.  Over two years of data including that from Facebook was released through the legal system in a Colorado court case where injured people were suing Wal-Mart.  I haven’t examined the details  but this is also something to bear in mind.  Not only can some government employees get to peruse what you’ve done on Facebook but others can too.

Crackers (often called hackers in the popular press) have in the past been able to grab a lot of information.  An exploration of the Facebook interface will show an alert user places where the system is weak and the bad guys might be able to get in.  I wouldn’t be surprised if everything you have on Facebook is being siphoned off by unsavoury characters.

Programming.  You can see what programs can access through applications like Facebook Mirror, or go in yourself and see what is revealed by the programmer interfaces.

To wrap up, there’s several ways that your data (like your profile) is shown to others.  There are ways to control this but it takes some work.  It also changes every few months so you need to repeat from time to time.  If you object to how it works you can always delete part of what you’ve already published.  Facebook, it seems, isn’t going to help you understand this.  You need to find out for yourself.  If Facebook’s value to you exceeds this effort there’s no need to delete your account.

This is the result of some online research and my best guess as to what’s going on.  I didn’t want  to research it myself, but found no option!  I haven’t validated it all thoroughly.  Caveat Emptor.  If you’re using it for  things that matter to you check your facts first, they change often.  Mike Gale.

HTML email not displaying to some users (System.Net.Mail)

I experienced a problem when using System.Net.Mail with .NET 3.5.

Mails with both an HTML message and a plaintext version weren’t always showing the HTML version, as I intended.  In Outlook and some Webmail services, I got the display I wanted.  In others I didn’t.

PortionOfAlfsLinqEmailOutlookViewViaLitmus

For example two Webmail systems,  GMail and Yahoo, showed plaintext and not the HTML.

The system was coded using AlternateViews in the MailMessage of System.Net.Mail.  The coding was influenced by online code samples.  In these the HTML message (which has an embedded image) was added first followed by the plaintext.

The intent is to show HTML (plaintext is a fallback).

Testing showed that the message had both payloads in the raw source.  (You can easily check this in GMail for instance.)

There are many posts about this same issue on the Internet (PHP, Cold Fusion, .NET…).  Not one that I saw had an answer to the problem.  Some reflected months of frustration.  That convinced me there was no solution.  (Foolish trust in the wisdom of crowds I guess!)

Being convinced there was no easy solution I still scanned the email RFC’s, 2822 and 2046, to see whether I’d missed anything.  I soon found something that looked like an answer.  Here’s a portion from 2046

Systems should recognize that the content of the various parts are interchangeable.  Systems should choose the “best” type based on the local environment and references, in some cases even through user interaction.  As with “multipart/mixed”, the order of body parts is significant.  In this case, the alternatives appear in an order of increasing faithfulness to the original content.  In general, the best choice is the LAST part of a type supported by the recipient system’s local environment.

Elsewhere it notes that it’s a crazy way to define things, but that it’s more compatible with older mail readers (so the past wins not the future!). I didn’t check but I imagine the readers referred to are from the 1970’s or earlier.  (These RFC’s have a long history of revision and number changes, so whatever the date of 2046 the message was likely originally written earlier.)

I rechecked the online documentation for the code I was using.  I didn’t see any explicit mention of the issue.  (Though an example, that I now found, added the views in the order that I wanted, plaintext first, HTML second.  No comment about why, that I saw!)

Changing the order fixed the problem.  Now all mail readers, that I tested, showed HTML.

Vinceremo!

Lessons

Here’s a couple of conclusions, from this incident and the project it is part of:

  1. If you’re doing both HTML and plaintext in an email.  Add plaintext first, HTML second when using System.Net.Mail.  With other systems make sure that the HTML payload occurs later in the raw message stream. (Assuming you want HTML to show up by default.)
  2. If your images are unique to an email, embedding (as opposed to linking) is probably best.  That way you have no need to store a lot of personalised images on your server.
  3. Using both HTML and plaintext is not a bad idea, it can reduce your spam score.  So even though you might not really want the plaintext there are other reasons for putting it in there.
  4. If you’re writing specifications that might last for a long time please consider designs that make sense in the long term.  Short term compromises might be around a long time and if they’re irrational they will waste a lot of programmer time.  Least surprise is a good design philosophy.
  5. Implementations that blindly follow a specification, elderly or not, are sometimes going to cause problems, especially if competing implementations do it differently.  A lot of people have got used to systems that let them prefer HTML.  The specification in fact suggests that, but doesn’t require it.  (There’s a de facto standard, please satisfy it if you are making the choice.)
  6. Documentation should be put together by somebody who knows the details of the underlying code (and why it’s that way) and tells programmers what they need to know. I’m guessing that any thorough programmer who’d use this API would know the issue.  We need that insight in all our documentation.  (If you’re writing the documents please give us the collateral information.  It could save programmers months of unwanted stress.)
  7. One post describing the issue was from somebody who later discovered the solution.  The earlier post was not changed to point to the answer.  Please update your posts. I know it’s easy to treat them as fire and forget, that can do the community a disservice.
  8. Don’t trust the Internet.
  9. Free services on the Internet might not be designed the way they would for a paying customer.  Careful of these things.  (I’ve been bitten by several free forum systems.  They one day realise that it’s no worth doing, they pack up and go away, taking your content, and membership lists with them (unless you saved).  Will the free emails go the same way?)

I like using .NET, but issues like this worry me.

If you want to test the system I’m talking about it’s at this URI at least for the immediate future (2010-04-20).

Suggestions for LINQ tool to use

About a tool that may help a developer using LINQ with his .NET coding (version 3.5 or later).  Helps to choose between Linq to SQL, Linq to Entity Framework and Plinqo.  It’s a web page questionnaire that emails a suggestion.

I recently struggled when using LINQ in a .NET 3.5 programming project.  I’ve used LINQ before and really like it.  I was convinced, by some web postings, that my original tool (LINQ to SQL or L2S) should be replaced.  I spent some time on the other approach, decided against it (for this project at least), and went back to L2S.

Then I saw that the choice was even wider than those two.

QnrToLetter20100108

Isn’t it always the case?  The web is a place of opposing opinions, half formed opinions, wise opinions and dumb opinions.  All mixed up and hard to sort out.

This area, choosing a LINQ tool, turned out to be pretty bad.  Digging showed me a lot of things like a petition, by about a thousand programmers, to Microsoft and the world at large basically saying, use L2S not L2EF (LINQ to Entity Framework).  It also showed me people who were enthusiastic about L2EF and advocating it’s use.

I often find these choices annoying.  They seem to eat up too much time.  You see a well written, convincing case for one choice, then you spot a difference from what you need.  The difference might look small and unimportant, but it’s enough to change the best choice completely.

Then you have to weigh up several options that are all good enough, but each has trade-offs that you don’t like…

I decided to create something that would have helped me when I made the choice.  Would have saved me a lot of time.  A web based system that helps make the choice.  It includes features that I like:

  1. One web page.
  2. You still do your own research and that research counts.  The system just guides you, it doesn’t take over.
  3. Uncertainty is OK, it just works.
  4. You’re not squeezed into a few radio button choices.
  5. You get to keep the results.  (So you can ponder over them and maybe do it a second time, without losing your notes.)
  6. It doesn’t get you onto some spammy mailing list.

Enough of that.  It’s at one of my web sites.  If you are facing this choice give it a try, it just might help you.  (Currently it’s free.)

Site under maintenance at 2010-04-14 11:20 Sydney time.  Back at 2010-04-14 12:50.

Thanks for those who have tested the application to date.  Your work has been useful and is appreciated.

Suggestion1

Note:  I don’t work for or represent any of the teams or companies creating these tools.  Codesmith presented me with a license for their product which I used to further my research into the Plinqo option.  Thanks very much Codesmith.

Working at DARPA and online questionnaires

I was reading a New Scientist book review about working at DARPA.

It struck me that it would be interesting to see two things:

  1. What other New Scientist readers thought about the jobs at DARPA
  2. How easy it is to create an online survey

So I ran an online search for survey tools.  I took the third one (mainly because I couldn’t see how it ran without giving it a try) and created a survey.  I put a j.mp link (which is the same system as bit.ly) to it on the New Scientist site and went away for a day.

That took a few minutes.  The survey questions were created on the fly.  If you did this regularly it would be quicker, and automation might be possible.  (Respondents can pick as many answers as they want.)

When I came back to look at the results, the next day, 22 had participated in the survey.  Here’s the results:

WorkAtDARPASurveyDay1B

The next day (2009-11-14) I captured the results again.  This time there are 27 responses.  Here’s what they look like (as computed by the service).  The percentage numbers are worked out in an odd way.  I make the “I have an ongoing…” question ticked by 48% of participants so multiply the percentage column by about 2.4 to get % of respondents.

The results look much the same though we now might have one response from somebody working at DARPA or a similar place.

ProcessedReducedImageOfSurvey27In20091113

Commentary:

  • This isn’t a scientific survey as much as exploring an idea to liven up the web.
  • The sample size is small, 27 respondents.  The readership who responded seem overwhelmingly to like the idea of what DARPA does.  A lot of them think they would make a good contribution.  One of the other responses is basically saying he would like to do the unclassified work and would not be prepared to do any classified jobs.  A fair number indicate a willingness to do this for free, I’m not sure how that would stack up if they were actually offered the opportunity, but it does seem to show a high level of enthusiasm.
  • I would prefer a different analysis to that automatically calculated.
  • The free membership at freeonlinesurveys doesn’t offer a database/spreadsheet data download.  This is a pity but the capture of the screen image isn’t too bad.  For a fee you can get a data download!  The free version gives 50 respondents and it works for 10 days.  That is too restrictive for many purposes.   (I had a look at the HTML generated for the graphs.  I was not impressed with the way it had been done.)
  • I found a few reviews of these poll / survey systems after I did this.  They offer a range of features.  Some of these are: freeonline surveys, PollDaddy, Question Pro, Survey Monkey, Zoomerang, SurveyGizmo, SurveyPro…  If you were serious  it would be worthwhile to look at their features more thoroughly.
  • I find much online content falls far short of what can be done online.  It’s derived from ideas of newspaper and magazine publishing, without recognising the capabilities of the web.  The web can do so much more than printing.  Interactive pages, polls, some real computation power can be unleashed through a web interface.  I’d love to see more of this smarter web in future.
  • Conclusions:  *In a few minutes a survey can be set up to gather worthwhile information.  *This sort of thing is more attractive to me than plain text.  *It takes longer to write this up than it does to create the survey.

Testing Windows Live Writer

This is my little test of WLW.

It seems to me that this tool might be a reasonable front end for CMS system that edits something other than a blog.

It will be judged on:

  1. Management interface for blog content.  For example deleting the test blog.
  2. Archiving of content.
  3. Comment management tools and approach.
  4. Quality of the markup.
  5. X/HTML Design of the pages, use of markup, CSS etc. that I would be proud of if I’d done it myself.
  6. Performance of the pages.
  7. Integration with existing content and tools.
  8. Quality for indexing.
  9. Documentation of the system.

Some of the presents on Christmas day overseen and guarded by Angus.

AngusAndThePresents

One of our Christmas day visitors at the front door.

 BirdlifeAt11Furber01

Observations

  • Inserted pictures are resized by default.  No way yet found to get rid of this nonsense.  The pictures are also messed with, by default there is some drop shadow effect, again don’t know how to stop this automatic silliness.
  • Spelling is indicated by red squigglies.  This is good.  Right click gives a list of suggested spellings.
  • Can switch between blogs on different blog systems.  Can cut and paste text.  Images fail (one attempt). 
  • Need to save between switches as blog is not automatically saved, so work can be lost, when used in that way.
  • This also failed on the first attempt.  (Connection closed…)  Check version of Windows Live Writer, maybe there’s a more recent version!!  This is 12.0.1366.1026.  There is a 2007-12-12 article describing an update to 12.0.1367.1128.  Unfortunately the knowledge base article links describing this seems to be dead, after several attemps it comes live, maybe a 3 network issue?  I installed the msi I obtained on 2007-12-26.  It does not appear to be that latest version, version number remains unchanged.  (Note the an msi is needed as the other installer is brain dead and refuses to install on Windows 2003, duh!!)  Maybe the update is installed by Windows Update:  try that starting at about 11:00 Sydney time on 2007-12-31.  Four times it fails attempting to contact the update site (Error number: 0x80072EE2).  This is a waste of my time.  Forget it for now.
  • 2008-01-02 today update started working again, without prompting, and it updated wlw, lets see if it now uploads.
Follow

Get every new post delivered to your Inbox.