« Previous | Main | Next »

New BBC Comments Module: Technical Overview

Comments

Share this page

Post categories:

Richard Summers | 21:07 UK time, Monday, 28 March 2011

I'm a developer of the Audience Publishing Services team in Programmes and On Demand in BBC Future Media. As Jessica has explained last week we rolled out a new comments product to all BBC blogs.

As a recent addition to the APS team I was tasked with revamping the commenting system and bringing it onto the BBC's Dynamic Web Application Platform (the Platform - PHP with BBC flavoured Zend framework).

The old system comprised of a set of API's which would allow developers to interact with the APS backend system. This would allow them to retrieve a list of comments (we call it a comment forum), and post comments back to it.

For ease of use, with products such as blogs, there was a set of SSI includes which would be embedded within a page which would pull in the necessary assets, forms etc and do all the API calls necessary to make it all work.

So why change?

The old architecture was basically a set of API's to the APS backend (which is tried & trusted), but the frontend implementation was difficult to implement and was totally geared towards working on the older SSI based servers, not the Platform.

We needed an easy way of adding comments to pages on BBC Online which would work on the Platform, would be backwards compatible with the older servers, and one that would stand up to the high traffic demands of clients such as BBC News and BBC Sport.

So what did we do?

Firstly, whatever system we built, it had to interface with the APS API (our source of comments and comment forums). We needed something on the Platform which would act as a wrapper for these API calls, so we build the Comments Service. The Comments Service is the bridge between the Platform and APS. Functions are called on the Platform using the Comments Service, which in turn calls the relevant APS API calls, which then returns JSON back to the Comments Service, to return it back to the application which called it initially.

It's not unusual for us to see peak levels of traffic of up to 500 transactions a second for News pages using Comments, so the Comments Service makes use of Memcached, this enables us to cache the response from APS for a short while, to relieve pressure on the APS Service at times of heavy load.

Graphs showing caching of bbc comments module on 21st March 2011

 

An important thing to note in the diagram above (taken from our trial of Comments on BBC News Stories), is the scale of the y-axis for each of the graphs. Even though the requests per second from the Comments Module to the Comments Service peaks at nearly 120 requests/second (requests for a list of comments), the requests to the Comments Service to the APS Service never goes above 10 requests a second. This enables us to cope with the large spikes in traffic and scale the module to meet the demanding needs of the bbc.co.uk domain.

The next thing we needed was something to handle the presentation of comments, the forms needed to interact with comment forums (posting a comment) and deliver the assets and javascript to enable a rich user experience for the whole thing. This, is called the Comments Module.

The Comments Module is a small application, which follows the design pattern of the BBC Module (others BBC Modules include Barlesque - the masthead and footer on all BBC pages). This allows it to be included in any of our Platform pages with only a few lines of code. To include the Comments Module on the old SSI based servers, we use the Comments Module as a full application, and call it via a dedicated URL using SSI. When the Comments Module is called in this way, it returns all the HTML, images and javascript needed to embed it into a page, so wherever the developer adds the SSI call to the URL of the Comments Module in their web page, the Comments Module will appear in all it's glory, already styled (in it's default appearance - which can be overridden with CSS), ready for use (including integration with our login system BBD iD).

The Comments Module uses the Comments Service to retrieve the comment forums from APS, and then renders them using the BBCs templating system - Spectrum. This takes care of rending the correct HTML, script, CSS tags etc.

So whats new in the new BBC Comments Module?

So, aside from a UI refresh (which at the moment is mainly visible on BBC News), we've added a few things which you might notice:

Speed Bumping

When this is enabled on a site, or comment forum, it enforces a minimum time interval between posts to the same comment forum by the same user. This is like a cooling off period between posts.

Ajax Pagination

When paginating between different pages of comments, if you have a javascript enabled browser with javascript turned on, all pagination happens using Ajax, making it load faster and increasing bandwidth efficiency.

Ajax Comment Posting

As with the Ajax pagination, posting comments can now occur using Ajax (specifically the BBC javascript framework Glow), meaning a faster response for users, and again a more efficient use of bandwidth.

BBC comments highlighted

Comment Highlighting

We found the old way of linking to a comment could be confusing, as in a long list of comments, it wasn't always obvious which one had been linked to, so we've introduced highlighting for linked comments. See here and here.

Comments Per Page

We've reduced the number of comments per page to 100 for all blogs (from 500). This reduces page size, which is better for bandwidth and page delivery times.

HTML in Comments

We allow a small set of tags to be used to style your comments on BBC Blogs. These are: <blockquote>, <em>, <li>, <p>, <pre>, <q>, <strong>, <ul>, <ol> and <b> These are not currently available for use on BBC News comments.

The Future

We've got a full roadmap for new features to be introduced in the coming months, i'm looking forward to telling you all about them, as we introduce them.

We're always interested to hear your thoughts, please use the comment box below!

Richard Summers is lead developer, User Publishing Services, Programmes and On Demand, BBC Future Media.

Share this page

Comments

  • Comment number 1.

    One great thing I've noticed about the new comments module is that it responds to my browser's accept-language header when displaying month names. So when I'm using an Irish browser, the name of each month is automatically translated. I hope this is sign of things to come and you start doing it everywhere.

    However I have noticed a small bug, if I send a http accept-language header specifying Scottish Gaelic, the comments fail to load. Though this probably isn't a comments specific bug, I've noticed similar anomalies all around the bbc site (for example buzz). Not that I mind, I don't speak Scottish Gaelic.

    Complain about this comment (Comment number 1)

  • Comment number 2.

    This comment was removed because the moderators found it broke the house rules. Explain.

    Complain about this comment (Comment number 2)

  • Comment number 3.

    Before this new change, only the first ten entries in each monthly archive list had the number of comments shown against the entry, which wasn’t very helpful, but the effect of the new changes seems to be that the number of comments shown against each blog entry has now disappeared altogether (at least for pre-March 2011 listings), which is even worse.

    Russ

    Complain about this comment (Comment number 3)

  • Comment number 4.

    On some early blogs, the comments are not numbered. Do you plan to retrospectively number them?

    Russ

    Complain about this comment (Comment number 4)

  • Comment number 5.

    For a long page of comments (> 100) if I refresh the browser on the last page to make the last few new comments appear it returns to the first 100 comment page, particularly annoying if you are following a debate with multiple pages. Perhaps increase the resolution to 200 comments?

    Complain about this comment (Comment number 5)

  • Comment number 6.

    Increasing the resolution won't help, unfortunately, marko. The underlying page architecture is wrong, and it always has been.

    Russ

    Complain about this comment (Comment number 6)

  • Comment number 7.

    5. At 10:19am on 29th Mar 2011, _marko wrote:
    For a long page of comments (> 100) if I refresh the browser on the last page to make the last few new comments appear it returns to the first 100 comment page...
    ------------------------------------------------------------

    Just click on the time shown next to a recent comment and then refresh will take you back to that comment even if it is not on the first page.

    __________________________________________________________________

    It is not that surprising that the comments module gets to its busiest at lunch time. It would seem a good idea to schedule enough moderators to cover this busy period. Sometimes it seems they have gone to lunch when they are most needed and this delay to comments prevents the natural flow of ideas.

    Complain about this comment (Comment number 7)

  • Comment number 8.

    This comment was removed because the moderators found it broke the house rules. Explain.

    Complain about this comment (Comment number 8)

  • Comment number 9.

    Re your HTML in Comments section, could you please make an extended version of this a standard part of the blogging help pages?

    Since you started supporting the UTF-8 character set, there have also been differences in the HTML entities supported. A full list of supported characters would also be helpful.

    In particular, when the new look was first introduced, us bloggers could provide help for other bloggers by demonstrating HTML code. Right now that's impossible because we can post individual GT; (>) characters but not the LT; character.

    Inability to post the LT; character will be difficult on some technical/economic blogs and cause a lot of head-scratching by non-techies.

    Worse, it means that web techies like me can no longer help them!

    Previously we could demonstrate using the LT; and GT; entities. Is there a workaround?

    Complain about this comment (Comment number 9)

  • Comment number 10.

    The comments module will always be busy at lunchtime and as someone mentioned in an earlier comment moderators should be out in force during this time (maybe they can schedule an early lunch (cant imagine that between 11 and 12 is too busy) :)

    Great post, its good to get an insight of how you are dealing with existing problems.

    Thanks,
    Scott McLay
    Editor of Scott McLay's SEO Blog

    Complain about this comment (Comment number 10)

  • Comment number 11.

    Another bug that I've noticed is that the username at the top of the comments module (where it says "you are currently logged in as x") links to the wrong page for me.

    The usernames at the top of each comment link to the correct place.

    Complain about this comment (Comment number 11)

  • Comment number 12.

    #11 lucas42
    "Another bug that I've noticed is that the username at the top of the comments module (where it says "you are currently logged in as x") links to the wrong page for me. The usernames at the top of each comment link to the correct place."

    Snap! I wasn't certain whether it applied to everyone or just to me because my username is no longer the same as my displayname, but it seems to.

    To clear any uncertainty, does your username/displayname at the beginning of the comments section just after the "You are currently signed in as" link to https://www.bbc.co.uk/blogs/profile.shtml?userid=27931 I wonder? If to the same userid, it's clearly a generic fault. If to a different one, maybe you too have a different username to your displayname.

    FWIW:
    ● The link to lucas42 (or You as you see it) is to https://www.bbc.co.uk/blogs/profile.shtml?userid=883812

    ● The link to Barbazenzero (or You as I see it) is to https://www.bbc.co.uk/blogs/profile.shtml?userid=12508774

    Complain about this comment (Comment number 12)

  • Comment number 13.

    Richard,
    "We allow a small set of tags to be used to style your comments on BBC Blogs."

    I'm disappointed that you haven't passed by to reply to any of these comments, but live in hope.

    Your list of tags supported doesn't include the a tag, BTW.

    The blockquote tag didn't fail on the last version of the blog site, and nor I believe did the pre tag, but neither did they work, whilst the ul and ol gave odd results. As there is no publicly available test area, there follows a test of each...

    This text is within a blockquote tag.
    It should appear indented.


    This text is within a "pre" tag.
    It should appear in fixed-width font
    and  allow  multiple   spacing between   characters.


    The following is a test of the ul tag which should produce a bulleted list of li tags.

    • first

    • second

    • third



    The following is a test of the ol tag which should produce a numbered list of li tags.

    1. first

    2. second

    3. third



    In preview mode, all work except that the ol tag produces a bulleted list instead of a numbered one, but previous implementations differed in "preview" and "posted" modes, so we will have to look at the post-submit situation.

    Complain about this comment (Comment number 13)

  • Comment number 14.

    PS to my #13 above, the pre tag is potentially very useful if you need show tabular columns, as the table tag has never been supported on BBC blogs, viz:


       Brighton Pavilion 2010
     Candidate   Party   Votes
     Lucas       Grn    16,238
     Platts      Lab    14,986
     Vere        Con    12,275
     Millam      L-D     7,159
     Carter      UKIP      948
     Fyvie       SLab      148
     Kara        Cit        61
     Atreides    Ind        19
     Total votes        51,834

    Complain about this comment (Comment number 14)

  • Comment number 15.

    Preview and basic html tags are not loading or functioning on a number of blogs.

    Russ

    Complain about this comment (Comment number 15)

  • Comment number 16.

    Hey guys, thanks so much for the feedback, it really is appreciated.

    Barbazenzero & lucas42 - thanks for pointing out the profile link bug, we should have a fix for that deployed in the next few days. Barbazenzero - you're also correct about the a tags, we do support an a tag, with an href attribute, and a title attribute. We are also working to address the ol tag issue.

    _marko - Yes the retention of the current page in paginated comments this is something we're addressing.

    And to Scott Mc and the rest of the contributors, thanks again for the comments and feedback!

    Rich

    P.S. #9 Barbazenzero - will get the GT; thing looked at, I know how useful that is, and appreciate techie help you guys provide to non-techie users. Rich.

    Complain about this comment (Comment number 16)

  • Comment number 17.

    Richard,

    Thanks for the response. I will pass by again soon to see how it develops.

    The current version is the best so far at supporting usable HTML and it would be a pity not to sort out the "rough edges".

    Complain about this comment (Comment number 17)

  • Comment number 18.

    This comment was removed because the moderators found it broke the house rules. Explain.

    Complain about this comment (Comment number 18)

  • Comment number 19.

    Displaying the GT character is no problem in BBC blogs, but displaying the LT character still seems an impossible dream.

    Workarounds using HTML entities still fail, as do using the UTF-8 character numbers for LT, viz:

    Testing GT; should be & # 6 2 ; omitting the intervening spaces gives: >
    Testing LT; should be & # 6 0 ; omitting the intervening spaces gives:

    Complain about this comment (Comment number 19)

  • Comment number 20.

    Has the non-operation of the basic html tags been fixed?

    Russ

    Complain about this comment (Comment number 20)

  • Comment number 21.

    Russ,

    As far as I've seen the only tag which used to work but does no longer is the i tag, but the original em tag does the same job. Note that support is not guaranteed for the b tag, although at the time of writing it's still working view source. However, the original strong tag does the same job.

    What other tags did you have in mind?

    Complain about this comment (Comment number 21)

  • Comment number 22.

    This comment was removed because the moderators found it broke the house rules. Explain.

    Complain about this comment (Comment number 22)

  • Comment number 23.

    Thanks Barbazenzero, but I’m getting some strange behaviour. I can post plain text ok. A message with some html formatting appears fine in preview mode in this blog, but as soon as I try to post it, I get a message “We're having some problems posting your comment at the moment. Sorry. We're doing our best to fix it.” and the comment box locks up and becomes unavailable for further editing. (This message may or may not be associated with the presence of html components of course. It’s impossible to tell unless someone at the BBC tests to see what is going on.)

    Whilst the blockquote function resolves correctly in this blog preview mode, it does not resolve in other BBC blogs’ preview mode, e.g. Radio 4. However, in those other places, although the preview mode does not show formatting, the html-stripped comment does get through the posting phase.

    There therefore seems to be some inconsistency in how the preview and/or html scripts are loaded or handled across the blogs. Hence my original question.

    I will try a third attempt at putting in some formatting in a following post.

    Russ

    Complain about this comment (Comment number 23)

  • Comment number 24.

    One of the problems here of course is having to wait for the non-operational 300 second counter to operate before being able to post a new comment. The result of being told to wait for 300 seconds is that the content of the comment box locks up and becomes unoperational as well.

    Really, it's total carp.

    Russ

    Complain about this comment (Comment number 24)

  • Comment number 25.

    Nope, I can't post anything with html components at the moment. (I'm using Safari 5.0.4, but I can't believe the parser for this comment box is browser-specific. I might try something later in Internet Explorer to see whether that makes any difference.)

    Russ

    Complain about this comment (Comment number 25)

  • Comment number 26.

    #25 Russ
    "I'm using Safari 5.0.4, but I can't believe the parser for this comment box is browser-specific. I might try something later in Internet Explorer to see whether that makes any difference."

    I'd find it hard to believe that it could be browser-specific. Posting HTML source code works fine for me in MSIE, Firefox & Chrome (from which I'm posting this message).

    Are you using an HTML editor? If so, you might have to open it in code view to strip out styles, fonts and the like, which can confuse the parser. Try the KISS principle of building up to complexity, and remember that the parser does add a br tag to the end of each physical line, so make sure you have line wrapping on.

    Complain about this comment (Comment number 26)

  • Comment number 27.

    [Follow-up formatting attempt]: bold, italic using i, italic using em, and

    blockquote

    and an a ref (to the BBC front page).

    Russ

    Complain about this comment (Comment number 27)

  • Comment number 28.

    Russ,
    Good try, but you're obviously having trouble with the a tag. See https://www.w3schools.com/tags/tag_a.asp

    Complain about this comment (Comment number 28)

  • Comment number 29.

    Barbazenzero - no I'm using straight keyboard input, or pasted from a plain txt file, i.e. without styles, fonts etc.

    #27 seems to have worked, at least partially. What was confusing me was that the preview parser will accept a url without quotes, but the posting acceptance-stage parser will not, and will lock the comment box content and terminate the posting process.

    The a ref parser is a bit strange: the parser accepts both an 

    a ref=

    opening tag as well as an 
    a href=
    opening tag.

    (Not sure how the above paragraph will appear in post mode: the pre tags introduce an inconsistent number, either zero or one, opening/closing line breaks.)

    There seems to be no problem with the GT character >, either straight from the keyboard (shift period) or escape-coded, and whether pre or non-pre tagged.

    What I can't do is a LT character, either straight from the keyboard (shift comma) or by using escape codes. (See the last line of your comment #19, where it is also non-existent to me.) Further testing in the preview parser indicates that the LT character, whether straight from the keyboard or escape-coded, will not appear even if it is contained within pre tags.

    I'll need to do a couple of test posts in the R4 blog area to check the differences as reported in #23 are still present.

    Re-checking i tags, the remainder of this sentence should appear in italics.

    Russ

    Complain about this comment (Comment number 29)

  • Comment number 30.

    Barbazenzero, yes, you're right, I should always be using a href format. (That wasn't the major problem actually - it was checking which convention for the url the BBC was now adopting. The preview parser was also putting me off, accepting any a tag immediately and putting it in bold, which I mistook for a correct parse.)

    I note i tags (which always worked previously in BBC blogs) no longer work for italics. I find the use of em and strong so archaic!

    Incidentally, it seems that any code the posting-stage parser doesn't like within a paragraph will hide any further content of that paragraph.

    Russ

    Complain about this comment (Comment number 30)

  • Comment number 31.

    Is there a character or word count limit to comments?

    Russ

    Complain about this comment (Comment number 31)

  • Comment number 32.

    Russ - comments on this blog don't have a character or word count limit.

    Thanks

    Complain about this comment (Comment number 32)

  • Comment number 33.

    Thanks Nick, but I found I had to split a long comment into three parts yesterday on the Radio 4 blog - the preview screen and post button just would not work if the comment was input as a single entry.

    I found I also had to rewrite the blockquote content because the Radio 4 blog doesn't seem to like those tags.

    Russ

    Complain about this comment (Comment number 33)

  • Comment number 34.

    This comment was removed because the moderators found it broke the house rules. Explain.

    Complain about this comment (Comment number 34)

 

This entry is now closed for comments

Jump to more content from this blog

About this blog

Staff from the BBC's online and technology teams talk about BBC Online, BBC iPlayer, and the BBC's digital and mobile services. The blog is reactively moderated. Posts are normally closed for comment after three months. Your host is Eliza Kessler.

For the latest updates across BBC blogs,
visit the Blogs homepage.

Subscribe to BBC Internet Blog

You can stay up to date with BBC Internet Blog via these feeds.

BBC Internet Blog Feed(RSS)

BBC Internet Blog Feed(ATOM)

If you aren't sure what RSS is you'll find our beginner's guide to RSS useful.

BBC Online in the news and blogs

Links to conversations and stories about the BBC's online activities. The links on this blog and its delicious stream are chosen by Eliza Kessler and Nick Reynolds. Follow @bbcinternetlinks on Twitter.

Follow on Twitter


About The BBC

BBC Internet Blog

bbc internet links

BBC iPlayer

BBC Red Button

BBC Blue Room

BBC R&D

Ralph Rivera

BBC Blogs and Boards

Find all BBC bloggers on the blog homepage

More from this blog...

Topical posts on this blog

Archives

Past twelve months

complete archive

Categories

These are some of the popular topics this blog covers.

Latest contributors

BBC © 2014 The BBC is not responsible for the content of external sites. Read more.

This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.