EMC Documentum and Microsoft SharePoint reveal a vast divide in both resolve and vision.
EMC will tell you that Documentum is the obvious choice for enterprise implementations managing very large data and content inventories. EMC begins in the basement (with disk and server) and provides infrastructure that content-able implementations can exploit for the end-users benefit. Documentum is first and foremost a server backend, independent of any given UI framework and can be viewed as a content aware RDBMS layer above the traditional data layer.
Microsoft developed SharePoint from the user back to the server. SharePoint is indeed an interface first and provides platform architecture designed to support the experience. Not surprisingly, the nuts and bolts of SharePoint rely on other Microsoft technology to perform. SQL server with the addition of file streams is somewhat equivalent to the EMC Documentum Content Server role. SharePoint however achieves the functionality in a single scalable layer as opposed to two in the Documentum world.
Business Logic
A few years ago EMC Documentum held the crown for providing best in class business logic integration via the Business Object Framework that allows integrators (consultants) to write and register specific Java classes to execute business specific (non-Documentum) logic triggered by library services events (save, write, lock, view, etc…). The BOF capability allowed implementations to directly extend the Documentum DFC API and is quite efficient.
SharePoint business logic is largely workflow driven and much more asynchronous in nature than the EMC BOF layer. Lately improvements in .Net and Microsoft SharePoint’s best friend K2 has begun to fully exploit the SharePoint vision and much more synchronous “composition” based business logic can be applied to event triggers specific to content type items, lists and webs in a fashion that does NOT require a heads-down programmer to create or maintain. Again, the implementation starts with the experience and the time a user spends extending SharePoint functionality is largely spent working through the consumption tool (Browser).
Documentum’s answer lies in the newish Process Engine and UI. Process Engine introduces the ability to perform no-code workflows (aka K2) that are (or at least can be) event driven and defined visually through a thick windows client. Future releases will be optimized to provide a more synchronous execution of successive automatic workflow activities and in time the functional offering will be a competitive K2 alternative. Due to the close relationship between K2 and Microsoft I don’t think we will ever see a Documentum compatible offering although this would provide great benefit to many clients.
Cloud Capability
Without doubt Microsoft SharePoint, especially 2010, is the clear leader with regard to “cloud-ability”. There is no shortage of hosting partners providing both SharePoint and Exchange in a multi-tenant environment for extremely low customer investment levels. Microsoft itself is hosting Exchange, SharePoint, Live Meeting, Communicator, and Dynamics CRM at incredible monthly per-user rates. From experience trying to offer Documentum Services in an operational model (no capital expense) to clients is an impossible challenge with regard to turning a profit. The largest stumbling block is not the sales model either (although change is needed). But rather the server architecture is not written with the cloud in mind. Microsoft on the other hand is now releasing Office/SharePoint 2010 expressly to improve (if not completely mature) the cloud model.
Virtualization is the Foggy Future
Only time will tell, but if history is a reliable forecast, EMC needs to be thinking complete rewrite. It’s time to reorganize the very good server and client functionality into a softer cloud focused architecture. As customers continue to strive to reduce operating costs the current trend to virtualize server infrastructures with VMWare and Hyper-V will move to a more cloud like implementation when we will finally let go of the server mentality. Within 3 years I would be surprised to see large companies accept new software that must be provisioned at a server layer.
There is a lot of talk on the web regarding Silverlight as a replacement for InfoPath in SharePoint implementations. In order to investigate the relative ease of implementation this simple Sudoku application was created in Silverlight 3 using Expression Blend and Visual Studio 2010. Four days were required with the bulk of that time consumed in a learning curve for XAML and the new development suite. Judging from the time required to generate the application EAC is now firmly in the camp that Silverlight will (should be) the most common implementation for SharePoint web parts in the future. No forms server required and Cloud (BPOS) friendly.
Note that the application can be installed locally through the context menu. EAC will be exploiting this feature in a new Expense Report application for SharePoint Online.
SharePoint Migrator v1.0 allows users to create Excel based lists of metadata and file references for loading directly into existing SharePoint lists. Migrator is designed to utilize existing SharePoint web services (_vti_bin/lists.asmx) to transport data and files allowing Migrator to function as a load tool for any SharePoint instance including SharePoint online and other cloud based SharePoint services.
The new tool can be used with either SharePoint 2007 or 2010 Beta.
With the upcoming release of SharePoint 2010 and Access 2010 the barrier to complex application development in the SharePoint platform will be simplified to point and click. Do you remember the days when an access DB could satisfy departmental solution needs? Those days are returning in 2010 and carry significant advantages to those who opt for SharePoint Online.
Simply building out application and complex reporting in Microsoft Access 2010 will yield the ability to “release” the data model, forms, etc… into SharePoint at the click of a mouse. Stay tuned for your IT costs to be reduced!
Below is an excerpt for the Access 2010 online help entry regarding “Web Databases” (aka SharePoint Applications):
Build a database to share on the Web
You can use Access Services, a new component of SharePoint, to build web database applications. This helps you:
Secure and manage access to your data
Share data throughout an organization, or over the Internet
Create database applications that don’t require Access to use
This article provides an overview of how Access Services works, and introduces you to web database design. For an introduction to desktop databse design, see the article Database design basics.
Overview
Access Services provides a platform for you to create databases that you can use on the Web. You design and publish a web database by using Access 2010 and SharePoint, and people use the web database in a Web browser.
How it works
When you publish a web database, Access Services creates a SharePoint site that contains the database. All of the database objects and data move to SharePoint lists in that site.
Note To build a web database, you need Designer permissions on the SharePoint site where you want to publish it. For more information about SharePoint permissions, see the See Also section.
Forms and reports run in the browser
Forms, reports, and UI macros run inside the browser. This lets Access refresh data on the screen without having to redo the whole page.
You can create a form to help people navigate your application. A new control, the Navigation control, makes it easy to add standard Web-style navigation buttons to a form for this purpose.
Note The Navigation Pane (the feature that you use in Access to browse the objects in a database) is not available in a Web browser.
Data is stored in SharePoint lists
All of your tables become SharePoint lists, and records become list items. This lets you use SharePoint permissions to control access to your web database.
Queries and data macros run on the server
All SQL processing happens on the server. This helps improve network performance by limiting traffic to result sets.
Share on the Internet
You can use SharePoint Online to publish your web database to the Internet. SharePoint Online is an Internet-facing SharePoint deployment, hosted by Microsoft. For more information, see the See Also section.
Note This service is not available during beta.
Create a web database
This section describes key new features and provides steps for the basic design tasks you need to complete to create a web database.
Before you begin
There are a few tasks that you should perform before you start to design your web database. Moreover, there are design differences between web databases and desktop databases that you should know about, espcially if you are an experienced Access developer.
Determine the purpose of your database Have a clear plan so you can make good decisions when working out design details.
Find and organize the information required If you line up your data before you start designing, you can avoid having to refit your design to accommodate unexpected data challenges.
Identify the SharePoint site you will use to publish You cannot publish at all without SharePoint. If you want to test your design in a browser while you design it (not a bad idea), you have to publish it first.
Plan your security You can take advantage of SharePoint security to control access to your web database. Plan security early so you can build it into your design.
Design differences between desktop and web databases
Some database features that you can use in a desktop database are not available with Access Services. However, there are new features that support many of the same scenarios as these desktop-only features. If you are used to using one of these features, you should familiarize yourself with the ways these new features work.
The following table lists the desktop-only features, and the new feature that helps support the same scenario.
Scenario
Desktop-only feature
New feature
Designing database objects
Design view
Enhanced Datasheet view; Layout view
Reviewing summarized data, such as sums, averages, and groups
Group functions
Data macros
Programming events
VBA
Data macros; New macro design experience with Intellisense
Looking up a record and display it in a report
Lookups
Navigate to a database object
Navigation Pane
Navigation control or other form element
Desktop-only features with no Access Services counterpart
Union queries
Crosstab queries
Overlapping controls on forms
Table relationships
Conditional formatting
Various macro actions and expressions
Note You can create many client objects in a Web database, but you cannot use them in a browser. however, they are part of the Web database and can be used in Access 2010 on the desktop. People can open the Web database in Access, and then use the client objects. This is an effective way to share a database, and also opens new opportunities for working together over the Web. SharePoint handles any concurrency issues.
Consider using a template
When you have determined what your application must do, consider whether a database template would work. Database templates are pre-built applications that you can use as-is or modify to suit your particular needs.
You can review the available templates from the New place in Backstage view. For more information about the templates that come with Access 2010, see the article Introduction to the Access 2010 templates.
Get started with a blank web database
Click the File tab, and then click New.
The File tab opens Backstage view, a new part of the Access interface where you’ll find commands that apply to an entire database, such as Publish to SharePoint. When Backstage view is displayed, the File tab label changes to Back.
In Backstage view, the New tab has commands for creating a database.
Under Available Templates, click Blank Web Database.
Review the proposed file name in the File Name box, and the path for the database file, listed just below. You can change the file name by typing in the File Name box.
To change the path, click the folder icon next to the File Name box to browse for a location to put your database file.
Click Create. Your new web database opens and displays a new empty table.
Design a Web table
Note You use Datasheet view to design a Web table.
When you first create a blank web database, Access creates a new table and opens it in Datasheet view.
After you modify and use the new table, you’ll most likely want to create more tables.
Create a new Web table
With your web database open:
On the Create tab, in the Tables group, click Table.
Select the field that has formatting and properties that you want to set.
On the ribbon, click the Modify Fields tab.
Use the commands in the Formatting and Properties groups to change the settings.
Add a calculated field
You can add a field that displays a value that is calculated from other data in the same table. Data from other tables cannot be used as the source for the calculated data. Some expressions are not supported by calculated fields.
Select a new, empty field.
On the ribbon, click the Modify Fields tab.
In the Fields & Categories group, click Calculated Field.
The Expression Builder opens.
Use the Expression Builder to create the calculation for the field. Remember that you can only use other fields from the same table as data sources for the calculation. For Help using the Expression Builder, see the article Use the Expression Builder.
Set up data validation rules
You can use an expression to validate input for most fields. You can also use an expression to validate input for a table, which can be handy if you want to validate input for a field that does not support validation, or if you want to validate field input based on the value of a different field.
You can also specify the message that is displayed when a validation rule prevents input, known as a validation message.
Set up a field validation rule and message
Select the field to which you want to add a validation rule.
On the ribbon, click the Modify Fields tab.
In the Validation group, click Field Validation, and then click Validation Rule.
The Expression Builder opens.
Use the Expression Builder to create your validation rule. For Help using the Expression Builder, see the article Use the Expression Builder.
In the Validation group, click Field Validation, and then click Validation Message.
Type the message that you want to display when input data is not valid, and then click OK.
Set up a table validation rule and message
Open the table to which you want to add a validation rule.
On the ribbon, click the Modify Fields tab.
In the Validation group, click Table Validation, and then click Validation Rule.
The Expression Builder opens.
Use the Expression Builder to create your validation rule. For Help using the Expression Builder, see the article Use the Expression Builder.
In the Validation group, click Table Validation, and then click Validation Message.
Type the message that you want to display when input data is not valid, and then click OK.
Create a relationship between two Web tables
To create a relationship in a web database, you use the lookup wizard to create a lookup field. The lookup field goes in the table that is on the many- side of the relationship, and points to the table that is on the one- side of the relationship.
Maintain data integrity by using data macros
You can implement cascade updates and deletes by using data macros.
Create a Web query
You can use a query as the data source for forms and reports. Queries run on the server, helping minimize network traffic.
For example, suppose you use a Web database to track charitable contributions. You want to see who donated money while an event was occuring. You could use a query to select the data and prepare it for use in forms and reports.
Note This procedure uses the charitable contributions template as an example. You can follow along if you create a new database by using the charitable contributions database template.
On the Create tab, in the Other group, click Web Query.
In the Show Table dialog box, double-click each table that you want to include, and then click Close.
In this example, double-click Constituents, Donations, Events, and EventAttendees.
Create any required joins by dragging fields from one object to another in the query design window.
In this example, drag the ID field from Constitutents to the DonorConstituentID field in Donations, and then drag the DonorConstituentID field from Donations to the ConstituentID field in EventAttendees.
Add the fields that you want to use. You can drag the fields to the grid, or you can double-click a field to add it.
In this example, add Event from the table Events, DonationDate from the table Donations, and Greeting, FirstName, and LastName from the table Constituents.
Add any criteria that you want to apply.
In this example, you want to limit DonationDate so that it falls between the StartDate and EndDate of the event. In the query design grid, in the Criteriarow under DonationDate, type >=[StartDate] And <=[EndDate].
Create a Web form
Forms are the main way to enter and edit data in your Web database, and are also useful for reviewing data. Forms run in the browser, helping optimize performance. When you open a form, your browser gets the required data from the SharePoint server. You can filter and sort the data in the form without needing to get data from the server again.
Tip For best performance, limit the records retrieved by your main forms and reports.
Select a table or query to use as a data source.
Note If you want to create an unbound form, skip ths step.
On the Create tab, in the Forms group, click one of the following buttons:
Form Create a simple form that shows one record at a time, using the object you selected as a data source.
Note If you are creating an unbound form, this button is not available.
Multiple items Create a form that shows multiple records at a time, using the object you selected as a data source.
Note If you are creating an unbound form, this button is not available.
Blank form Create a form that has nothing on it.
Datasheet Create a form that looks and behaves like a datasheet, using the object you selected as a data source.
Note If you are creating an unbound form, this button is not available.
Create a Web report
Reports are the main way to review or print data from your Web database. Reports run in the browser, helping optimize performance. When you open a report, your browser gets the required data from the SharePoint server. You can filter and sort data in the report without needing to get data from the server again.
Tip For best performance, limit the records retrieved by your main forms and reports.
Select a table or query to use as a data source.
On the Create tab, in the Reports group, click one of the following buttons:
Report Create a basic report using the object you selected as a data source.
Blank Report Create a report that has nothing on it.
Create a Navigation form
People need a way to navigate your application. Remember — the Navigation Pane is not available in a Web browser. For people to use your database objects, you must provide them a means. You can create a Navigation form and specify that it be displayed whenever someone opens your application in a Web browser.
Tip You may want to wait until last to create your Navigation form, so you can add all your objects to the form when you create it.
On the ribbon, click the Create tab.
In the Forms group, click Navigation, and then select a navigation layout from the list.
To add an item, drag it from the Navigation Pane to the Navigation control.
Note You can only add forms and reports to a Navigation control.
Add any other controls that you want to the body of the Navigation form. For example, you might want to provide search functionality across all forms by adding a few controls to your Navigation form.
Publish and synchronize changes to your application
Before you publish a Web database, consider running the Compatibility Checker. The Compatibility Checker examines your database objects for issues that would prevent them from publishing correctly. If no issues emerge, the Compatibility Checker reports that your database is Web-compatible. If problems are discovered, the Compatibility Checker creates a table that lists the issues.
Click the File tab.
Under Information about database_name, click Run Compatibility Checker.
Publish a Web database
Click the File tab, and then click Share.
Under Special locations, click Publish to SharePoint.
In the middle of the tab, under Publish to SharePoint, fill in the following:
In the Web Locationbox, type the Web address of the SharePoint server where you want to publish your database. For example, http://Contoso/.
In the Web Application Name box, type a name for your Web database. This name will be appended to the Web Location to produce the URL for your application.
For example, if the Web Location is http://Contoso/ and the Web Application Name is CustomerService, the URL is http://Contoso/CustomerService.
Click Publish to SharePoint.
Synchronize a Web database
After you make design changes or take a database offline, you eventually want to synchronize. Synchronizing resolves differences between the database file on your computer and the Web application on the SharePoint server.
Bharat Shah is the General Manager of the Microsoft Online Services Engineering division and is responsible for providing Microsoft’s business productivity software in the cloud. The video bellow is from Microsoft Channel 9 and discusses the BPOS engineering work that is ongoing.
EAC is now offering Microsoft Business Productivity Online Suite and related services. Sign up for a free trial or contact EAC for more information here.
Microsoft released beta packages yesterday for both SharePoint and Office 2010 (Office 14) to TechNet and MSDN. This version of the SharePoint server makes significant progress towards blurring the line between on-premise and hosted cloud implementations (internal or external) as well as large additions and improvements to collaboration and integration capabilities. With that said, I fully expect SharePoint 2010 to be as game changing as the release of MOSS 2007 as the industry moves to consolidate IT technologies and cut costs out of the infrastructure and headcount.
I will be installing the new code today in a lab environment and will report back with my take. If you need assistance in evaluating the new release please use our “Contact EAC” page to request support.
Microsoft follows a now familiar pattern with evolving technologies. Initial product releases are simplistic and target core functionality only. As the business model and market needs are discovered major releases transform the product into a power player. SharePoint has followed this recipe to a tee and the 2010 release should be considered the first highly mature composition (no-code) web based framework capable of supporting most enterprises in a cloud based model. Watch Steve Ballmer discuss the future of SharePoint 2010:
(More video available here.)
As I have just moved my own company infrastructure onto Microsoft’s full Business Productivity Online Suite (BPOS), I will be posting regarding our experiences in making the transition to cloud based SharePoint, Exchange, LiveMeeting and Communicator. I can say now that the pending release of 2010 has me focusing our business plan to help clients leverage a very capable, cost effective solution that should help remove IT complexity from non-IT businesses.
I can remember dealing with clients that were struggling with over 100 Lotus Notes/Domino servers just to keep basic collaboration and email up and running… These same clients were installing 40Meg thick clients to access the proprietary dinosaurs that were increasingly strangling their IT departments and slowing response to business needs. Sad, scary stories that led to many career “changes” in those organizations. Microsoft Business Productivity Online Services provides messaging, collaboration, customizable workspaces and infrastructure while promising:
“Microsoft® Online Services is built on the pillars of streamlined communication, simplified IT management, and business-class security and reliability. Each pillar supports the overall value that customers get: rich, flexible, reliable, and secure online communication and collaboration services that simplify IT management and support enhanced user productivity and business agility.
We’re ready to support your customers, wherever they want to take their business. Our stable and dedicated data center infrastructure can scale dynamically to meet their additional capacity requirements.” — Microsoft.
Very intriguing future for us all…. Is BPOS the new Web 2.0 Operating system? This is where I intend to move my business back end so that my current servers can be re-deployed for component and hosted complimentary web service offerings. Over the next few months I will post on our progress in this move. Stay tuned.
Several hours of my life Tuesday night were spent with my head in the clouds. I was attending the “un-conference” conference on cloud computing dubbed CloudCamp in Indianapolis. The interesting discussion – generated around topics such as the very definition of the cloud and even the appropriateness of its use for various implementations or components - got me thinking about what is available and what is still missing from existing frameworks.
Microsoft (not surprising) has made major investments in building out Azure (Microsoft’s cloud implementation) and has created a rich set of documentation to describe its functionality and architecture. Their offering is leveraged for many of Microsoft’s own applications supporting products like Live and Mesh (a fantastic tool).
What makes a cloud a cloud? If you are in the business of supporting corporate or commercial applications, you are beholden to some sort of hardware infrastructure. Five years ago that infrastructure was most likely dedicated hardware (DB servers, application servers, etc…). Pushing for economy of scale and better utilization, IT shops have been offering the layers of n-tier application frameworks as business services within the organization. First RDBMS (Oracle/SQL Server) clusters, then application servers like WebLogic and WebSphere, were transformed as business offerings that hosted like tiers across the entire application space of an organization. Later ECM platforms and other application frameworks were extended to serve the enterprise in general (Documentum and now SharePoint). Fewer administrators and shared hardware (better utilization in both cases) drove costs down.
New applications still required new hardware to expand the services in most cases and non-functional design and implementation remained a lengthy and costly part of deployments. Hardware requirements to support peak loads were often miscalculated and either fell short or wasted power and CPU cycles. Cloud computing brings the promise of eliminating much of this cycle and is beginning to deliver.
What is a Cloud?
Literally a cloud is a group of water molecules floating above that is just big enough not to condense and fall and yet just small enough to hold the present level of moisture…. Restated a cloud is an ever changing system of capacity that self regulates based on pressure and temperature. In our world of computing a cloud is much the same. We just substitute capacity for good old H2O. When the pressure goes up a natural cloud will likely deliver some of its moisture to us… The computing cloud will deliver more capacity when we need it as the pool of infrastructure resources is “detached” from any specific task. Hardware and software vendors are beginning to sell cloud specific modular packages to help the industry (and you) build out with less hassle. EMC, Cisco and VMware now offer V-Block, a combination of EMC storage and Cisco virtualized servers running VMware underpinnings (see http://online.wsj.com/article/BT-CO-20091030-715748.html).
How does this differ from a virtualized server room?
In a virtualized environment (VMware for sake of argument), physical hardware and logical servers are managed separately. An application like VMware manages the bare metal hardware and its resources as a pool of available capacity at a server level scope. Administrators generally create rules to align logical servers with physical machines or more generally to portions of the available capacity (provisioning). This sounds a lot like cloud computing to me and in fact is very close to the finish line and satisfies some definitions as long as the virtual server instances can be dynamically allocated and scaled horizontally. Many companies (most at this point) have moved to this virtual provisioning technique or are moving in that direction.
What else is needed to achieve an internal Cloud?
The difference between virtualized servers and a cloud is related to provisioning and scope. While VMware, Microsoft Hyper-V, Sun, Xen (open source hypervisor used by Amazon) and others provide technology to dynamically provision logical servers across physical hardware pools, they do not (typically) provision capacity for application layers on their own. Let’s look at the cloud from a different angle. Suppose I have a typical 3 tier application that runs on the web: Thick Client – Web Browser (Tier 1), Dynamic Web Application (tier 2), Database (Tier 3). The application can be broken down into data services, application services (logic) and UI services. All of these services have defined relationships and some design pattern to tie them together. In most cases the application services handle the integrations. Scaling this typical application usually requires separate activities for scaling out the application server (perhaps re-provisioning WebLogic nodes in a large cluster) and very complex re-provisioning of the backend database cluster. Often these changes require redesign and upgrade of SAN or NAS storage solutions, updating and retesting HA (high availability) and DR (disaster recovery) scenarios and backup/snapshot procedures. Many different corporate IT disciplines get involved, Program Managers, Unix Engineers, Windows Engineers, Application Specialists, End Users, etc…. When these problems go away, you have arrived in the clouds.
To achieve this goal a fabric must be woven above the provisioning layer that can dynamically align application spaces with runtime needs based on business rules provided at deployment. I believe that Microsoft Azure is on its way to becoming the most fluid environment available for running your applications and services and most closely fits the total model of cloud computing.
“The Windows Azure platform offers an intuitive, reliable and powerful platform for the creation of web applications and services.
The Windows Azure platform is comprised of Windows Azure: an operating system as a service; SQL Azure: a fully relational database in the cloud; and .NET Services: consumable web-based services that provide both secure connectivity and federated access control for applications.
Currently in Community Technology Preview (CTP), the services are free to evaluate through January 2010. We will begin charging customers on February 1st, 2010.” – Microsoft Windows Azure Platformweb site…
Notice that Microsoft views the cloud as an Operating System as a Service (OSAS). A significant point worth contemplating.
Most cloud offerings handily manage provisioning at a virtual machine level. However in most cases common facilities for handling provisioning and automatic scaling at a specific service level are still lacking. Also, the need manage state between the virtualized services loosely coupled to provide for such scaling still needs more standardized messaging and transaction boundries.
Cloud is good!
What if instead of using Oracle for a back end tier you had available an absolutely limitless capacity for data services abstracted from both vendor and hardware realities. One request for a result set or one million requests in an hour all produced with equal performance (from one application or thousands). Cost is variable based on your actual usage and there is NO requirement for internal staff or monitoring (assuming your application works). That would mean no load meetings to discuss outsourcing or personnel issues… Everyone on your staff could concentrate on moving forward and providing more competitive advantage in your market.
The benefit: If you need a new business or commercial application and can design it to run efficiently in the cloud (more on that another time) then you can build and release without much concern for capacity or hardware support of any kind. A good way to live! How far will we dive into the clouds in 2010? Only time will tell.
Drive innovation and efficiency into your business with Business Productivity Online Suite (BPOS) from Microsoft and support from EAC’s proven industry professionals [read more]
Under Available Templates, click Blank Web Database.
Review the proposed file name in the File Name box, and the path for the database file, listed just below. You can change the file name by typing in the File Name box.
To change the path, click the folder icon next to the File Name box to browse for a location to put your database file.
Click Create. Your new web database opens and displays a new empty table.
Microsoft released beta packages yesterday for both SharePoint and Office 2010 (Office 14) to TechNet and MSDN. This version of the SharePoint server makes significant progress towards blurring the line between on-premise and hosted cloud implementations (internal or external) as well as large additions and improvements to collaboration and integration capabilities. With that said, I fully expect SharePoint 2010 to be as game changing as the release of MOSS 2007 as the industry moves to consolidate IT technologies and cut costs out of the infrastructure and headcount.
