Jump to content

Tim Stephenson (Archant IS)

Members
  • Posts

    159
  • Joined

  • Last visited

    Never

Posts posted by Tim Stephenson (Archant IS)

  1. Can we have another round of comments about the forum's performance over the next few days?

    Specifically any comments on the following points would be useful:

    • Do the forums generally seem quicker (after 26/9) ?
    • Do you find them to be performing more consistently now - either faster or the same speed; but not varying wildly between running quickly and slowly?
    • For those on dialup, have you noticed any improvements over the last couple of days?

    Thanks once again for your assistance

  2. Baz,

    Thanks for this - positive feedback makes a nice change.

    As I think we've probably mentioned before, although these forums have been performing badly over time we are committed to improving the situation. Unfortunately these things just take time, effort and money...

    Monday's round of upgrades should help make things more consistent too.

    Any comments are welcome!

    Regards,

  3. To give a little more information, Living France's forums currently sit on a loadbalanced set of 3 webservers along with 150 other websites.

    We're in the process of looking at three sets of upgrades to this platform, with the common goal of improving performance for the end user.

    The first stage is in place, which is to trial content compression software. This should reduce the overall page size and reduce the time physically needed to download each page. The aim here is to improve page-load times. Initial results are looking promising but we'll continue to evaluate the benefits over the next month or so before reaching any conclusions. Ironically, dial-up users will probably see the most benefit from this!

    The second stage will be happening Monday morning, which is to upgrade the

    3 servers from their current 2Gb to 4Gb of memory each. We're hoping this will solve some of the performance inconsistencies you're currently seeing throughout the day by giving the webserver software more headroom to handle the load.

    Finally, we're planning a larger set of upgrades - adding more webservers, upgrading their operating system, and improving resiliance to failure elsewhere in the server farm.

    I don't have a timescale for the final point at the moment as we're currently in the process of determining what sort of server configuration is going to give us the best performance vs cost (dual 3.6Ghz processor servers vs quad 3.6Ghz processor servers; and looking at 64-bit systems vs 32-bit.)

    Alongside all of this we're also nearing completion of work on a replacement forum system which should resolve problems with things like the search.

    Regards

  4. [quote]Wonderful news I looked at the second forum http://forums.asp.net just for the hell of it and noticed in the Bugs & Problems with this website on the third post the following. does it ring ...[/quote]

    I have to say we haven't seen these search problems on the build we're running here. I'm not sure whether asp.net forums are running a customised or stock version, but ours doesn't seem to behave in the same way.

    Before deploying however I'll ensure that someone goes through the bug list shown on asp.net forums.

    Regards

  5. [quote]For almost a year now we have endured thus “sub-standard” forum software, regularly being assured the next update is now going to make everything better. We have been assured that IS have a “strategy...[/quote]

    In response to Deimos, here's an update on what's happening concerning the forum system.

    1) The current forum system has not changed because over the year its supplier has failed to deliver a version that is production ready and worth upgrading to. A number of scalability problems have been identified with the current version, none of which can be easily rectified without a ground-up structural rewrite. We have looked at the latest release of this software, and found it wanting in terms of performance and scalability.

    As such, we have continued to evaluate alternative pieces of software and have selected an industry leading product from Telligent Systems. This is a forum engine with proven commercial scaliability, and can be found powering large-scale forums in various areas.

    Some example sites:

    http://forums.xbox.com

    http://forums.asp.net

    http://forums.microsoft.com

    We have completed some initial migration work, and are looking to move all forums onto the new platform within the next few months.

    2) The current system was selected on the grounds that it was the best suited to our needs at the time. However business requirements have changed over the last year, and we've seen the volume of traffic grow beyond all expectation.

    3) Yes & no

    4) The forum is not directly funded by the magazine as it is an instance of a system shared across 150 other websites.

    Regards

  6. Update:

    traced to some extremely bad HTML code in "lizzy h"'s signature:

    The sig contained an html fragment,

    <TABLE id=HB_Mail_Container height=100% cellSpac

    and nothing else.. This would basically break any page into which it was placed.

    The sig's been cleared, and the problem sorted.

    Tim
  7. [quote]Quote: “I notice even TF (as an example mentioed by yourself) does not keep people informed the way you describe.”TF was raised by somebody else and don’t need to keep people informed about such issue...[/quote]

    I1, and how would you know what issues TF suffer from or have suffered in the past? According to your posts on here, you've only just joined their community!

    Running a single forum doesn't take much work. Running over 250 of them does.

    No forum solutions are ideal or trouble free due to how we need to them to operate. We can't support 250 separate installations of a forum system, with 250 separate copies of the forum code, and 250 separate databases.

    I'd expect you'll find that if you tried to push phpBB to these levels you'd encounter very similar problems.

  8. [quote]Baz, With respect, it is only because some members have been very vocal, that things did actually got a move on, regarding the continued and ongoing improvements to the site. If nothing had been sa...[/quote]

    Cheers Miki, although can I just point out that the only reason nothing happened for a while with the site was simply that we were tied up working on other projects (and the minor problem of Christmas / new year).

    The comments in the various threads have been extremely useful though in helping us to track down the cause of some of the problems, and in identifying some of the software's shortcomings.

    Keep 'em coming, though feel free to switch to PM's rather than public posts!

  9. [quote]I am finding it quite tedious to continually hear about about the shortcomings of this site. It is plainly obvious that Archant are trying to sort things out but the continual criticisms are not IMO ...[/quote]

    Thanks Baz.

    We do take the problems seriously, and this forum has had our continual attention for most of the last week.

    Most of the problems have now been resolved, and all being well the forum system is running smoothly. I have not been made aware of any more occurences of those "out of memory errors", but should any occur please PM me with a date and time.

    Many thanks

  10. [quote]Quote: “…“Last Page …” link to get to the post (or scroll through pages if there have been several posts and I cannot know where the first of my unread posts is). This requiring several pages to be ge...[/quote]

    To pick up I1's point - "But phpBB does not “fit-in” with Archant IT Group’s capabilities":

    LF is one of about 300 websites currently supported by Archant IS's Digital Media team, which as a combined portfolio generate 6-800Gb of traffic per day.

    All of these sites are running either ASP or ASP.NET code because as a team of professionally trained and certified Microsoft developers, this is the platform which we are currently best positioned to utilise. Our development team currently includes two of the first 1500 people worldwide to gain the premier MCSD.NET certification, and a number of other MCAD & MCP qualified staff.

    Unfortunately the enterprise environment is very different to that of a small business. Small businesses are typically able to select a platform based on the application they want to run, and are usually constrained in terms of budgets. This is why the PHP/LAMP stack tends to be so popular - it's all free software, allowing time and resource to be put into extending functionality or creating customised solutions.

    In the enterprise environment, while we're still constrained with budgets we are able to look more towards pre-packaged or commercial software, and utilise commercial server products. This is part of the reason we've been standardised on MS for several years, although in recent times this policy has been adjusted to mainly focus on the desktop environment. In terms of server products and Group applications we now aim to select the best solution that both matches our requirements, and fits the existing environment.

    When supporting this number of sites with a small development team, there's simply no point in starting to select random platforms for particular sites. If everything's based on the same core platform our development team is able to rapidly integrate new functionality, without needing to stop and spend time learning new skills to support 3rd party applications.

    So, I1, can we please agree to disagree over PHP vs ASP.NET, and this forum software vs phpBB?

    You're never going to like a product that runs on a Microsoft server, and we're highly unlikely to start ripping out established ASP.NET functionality, and replacing it with a PHP product (don't get me wrong, phpBB is a first class product - just unsuitable for our environment).

    You obviously dislike anything Microsoft, and by the sound of things think that everyone should be running their sites on PHP. While I can see where you're coming from here, as an enterprise function we don't have this level of flexibility as we have a large amount of existing systems to support and maintain, with which our sites and platform need to be closely integrated. At the moment, the best way to achieve this is through using MS products.

    This may change in the future, but to be completely honest, if we were to switch development environments we'd opt for something that could match the functionality and performance offered by .NET (Java, for example). PHP would be a backward step, and one which we would be unwilling to take.

    Should you wish to discuss this further, please feel free to PM me. There's really been enough public discussion on this subject now, and there's absolutely no point in continuing to bore the readers with an ongoing debate about the merits of one bulletin board system vs another.

    Regards

  11. [quote]Quote: “Thank-you, I1, for reminding us that nothing in IT is guaranteed”.Actually this is not the case atall. When we develop software we guarantee it. We guarantee that, should the customer identi...[/quote]

    I1, Fair comment!

    The problem we've got at the moment is identifying the exact cause of these errors. We cannot reproduce them on any other platform than the production webserver environment. Even when running on a development server, with a load simulator giving a greater load than the production environment (and on less powerful hardware) - the system runs perfectly.

    This is where we're concentrating our efforts at the moment as the production servers are running Windows 2K Advanced Server, as opposed to the 2K Standard Server in the development environment. There have been a few reported issues which seem to implicate 2K Advd server, with similar problems to these.

    If a few days of memory usage & allocation logging indicates that all's OK from the physical memory perspective, then the issues must be down to the differences between 2K Standard and 2K Advanced. If this is the case, we'll try rebuilding the servers as 2K Standard....

    I'll post an update once we've reached some conclusions.

  12. [quote]Time: 11.05hrs GMT Date: 15 Jan 2005 Tried to access the front page of the forum from scratch and got the following thrre times: Server Error in '/' Application. Exception of type System.Ou...[/quote]

    Following dialogue with MS tech support, it's starting to look like these problems may be being caused by issues with running IIS5/ASP.NET on Windows 2000 Advanced Server. Some configuration changes have been put into effect today that may help reduce the number of errors we're seeing.

    However, we will be running more tests and monitoring the servers over the course of next week to try and establish if this is the case. If found to be, we will then start looking at reconfiguring the servers with Standard Server rather than the Advanced edition.

    Regards

  13. [quote]Quote from Archant IS/IT: "We aim to select systems that provide the best combination of benefit / performance / resilience / supportability and compatibility ..."[/quote]

    Thank-you, I1, for reminding us that nothing in IT is guaranteed.
  14. [quote]This message cannot have happened. We have been told that the software and architecture are resilient and that all the problems have now been sorted.[/quote]

    I1, as you work in the software industry you will know that with software products - nothing is certain.

    I'm currently reviewing the servers' event logs to identify the machine causing these problems.

  15. Avatars should now be fixed. Upload tested with Jpeg, gif, png and bmp files using both Firefox and IE.

    If this doesn't work, please first try opening the image using IE and see what happens. If the image can be displayed correctly it should get uploaded ok.

  16. [quote]I havn't had any more time outs. The edit function is back, but I understand some people still can't see who's posting. The search 'function' is still terrible The quote function may as well not e...[/quote]

    Tresco, just to pick up your comment about the quote functionality.

    This is something that will be resolved in the next major release of the forums engine, which is due somewhere around March / April this year.

    Please let me know if anything else comes to light.

  17. [quote]Quote: “We aim to select systems that provide the best combination of benefit / performance / resilience / supportability and compatibility while meeting our requirements”I assume you consider this fo...[/quote]

    Quote: “We aim to select systems that provide the best combination of benefit / performance / resilience / supportability and compatibility while meeting our requirements”

    I1: "I assume you consider this forum system has such features as performance and resilience."

    For the previous, independent version, no. This is why we've moved it on to the same system driving our other forums, which does offer reasonable performance and is generally stable.

    Quote: “in high traffic environments the more scripting languages you have running on a web server the more problems you're likely to encounter”

    I1: "And we are having no problems here ? The loading on this forum does not represent high traffic. There are far larger phpBB sites around running decently (with decent functionality)."

    This forum is fairly reasonably loaded, in terms of forums. The overall forum engine however, is handling something in the region of 1.3 million posts by around 100K members. In turn, these applications run on a set of servers hosting other rather busier systems and websites generating approximately 6-9mb/s of continuous traffic. For those who work in terms of Gb per month bandwidth, taking peaks in traffic into account that's around 5-600Gb per day.

    As you have probably noticed, most of the reported problems (apart from the search) have been resolved.

    Quote: “To opt for a PHP solution would also require cross training of some of the development team to provide support for the system” – so, learn something new. Its what IT is about. phpBB can be installed an run by non-development people. My own company has a small phpBB system that was entirely installed, set-up configured and is maintained by a non-technical person with no help nor questions, nor involvement from any IT people.

    That's all great. However, unfortunately that's not how things tend to work in most enterprises. Systems need to be accessible to all members of a development team to provide cover in the event of absence or staff changes. We don't currently have any PHP skills in house as our internal development platform has historically been Microsoft based. This is a trend continued to the web, and as a result we've adopted ASP.NET for new web development. This doesn't mean it's the best language, but it is our current language of choice.

    Quote: “available for ASP.NET, which is our preferred platform” AND “we don't have a "Microsoft only" strategy” – you are clearly totally open to other architectures.

    Correct. Open to, and actively evaluating alternatives.

    I1: "A quote from a moderator “Living France's publishers' IT people, who insist that ASP.net (a Microsoft system) is the only way to go”.

    A quote from Tim Stephenson: “The Microsoft platform / environment is simply what our company is currently standardised on for software” Clearly no “Microsoft” strategy here !!!"

    As I currently represent Archant (Living France's publisher's) IT team for digital media, in this case I would suggest you take my comments in precedence to the Moderators.

  18. [quote]When I said funny, I meant funny ha ha, becouse I'd happened to catch that screenshot whilst IIS was re-booting.Just out of curiosity, how long has the server been up?I just checked mine - been up 13 ...[/quote]

    Sorry - never replied to this!

    The webservers (Win 2000 Advanced Server) have been running for around 30-35 days. We used to reboot them on a daily basis but have now switched to monitoring and automatically restarting the webserver process when required.

    The database servers (Win 2003 Server) have been up for around 165 days.
  19. I1,

    For your information, we don't have a "Microsoft only" strategy, although parts of our hosting environment are currently MS based.

    We aim to select systems that provide the best combination of benefit / performance / resilience / supportability and compatibility while meeting our requirements.

    For example, although we currently run Windows based web servers, our load balancers are from F5, the firewall definitely isn't ISA server, some infrastructure functions run on BSD/Linux, and the databases are split across SQL Server and MySQL. The only reason we're running Windows web servers is because our in-house development team are professionally qualified, Microsoft certified developers and we have a large legacy of ASP based code.

    This approach extends throughout the business. For example, rather than opting to remain with the established Windows/SQL structure for a new Group-wide ad-sales system, we've opted for the best available solution - which runs on Sun Solaris and Sybase (also on Solaris).

    To take your example of phpBB, while it is an excellent system for sites running php, in high traffic environments the more scripting languages you have running on a web server the more problems you're likely to encounter. To opt for a PHP solution would also require cross training of some of the development team to provide support for the system, and introduction of a potentially unstable element across all web servers. These were risks we were not prepared to take given the high levels of availability we need to provide.

    Unfortunately there are not many free, commercial grade forum systems available for ASP.NET, which is our preferred platform. The system we have selected has been running flawlessly across our other sites, but for some reason ran into problems when deployed separately to Living France. The France forums have now been migrated to the main forum system, and so far seem to be rather more stable. Other measures have been put in place to help identify / record when errors occur, and to aid us in tracking them down and resolving them. Also, given that this forum engine currently drives forums across something approaching 90 websites, the actual cost per site in terms of software licensing is trivial.

    Hopefully this will help explain the reasoning behind the decision to persist with this forum system, even in the face of problems.

    Regards

×
×
  • Create New...