I listen to a lot of cloud companies trying to pitch their products, enough that I can tell when someone is pulling my leg, or trying to steer me away from asking certain questions. Enough already. CIO's it is time to unite and stand up to this FUD...
If you are selling cloud, whether it's IaaS, PaaS, or SaaS, here are my 5 tips to help.
1. Cost is not why I'm going cloud
Cloud can cost less, but it can also cost more depending on what exactly your solution offers and what else I may need to buy to give me the same functionality. Office365 may cost me less than on site software, but it depends on what license I go with, how good my Enterprise Agreement was.
Frankly saving money is great and if there is a big cost savings, cool, but I tend to think of this as VoIP (voice over IP) years ago. When VoIP first came out and everyone was paying a lot of money for long distance, cost savings was a big deal. Today though long distance costs are so low trying to sell me on cost is a difficult pitch.
2. Performance
Cloud performance is tricky. I mean one of the ideas of the cloud is that it runs over the internet, which cloud vendors have no control over anyway. I suppose a big ISP could argue that if all of your sites and home users are on their network which also houses the cloud data center then their performance could be guaranteed. Most cloud vendors can't guarantee end to end performance, and trying to sell me on "transaction performance" is going to lead me to believe you are stupid, or you think I am stupid. Neither is going to help your case.
Now if the solution is entirely based in your cloud environment and very little user interaction is required, maybe this works. Big data analysis, gene sequencing, complicated simulations etc, may be some examples of where performance matters, but for me entering data into a spreadsheet, the performance issue is almost always going to be internet related which cloud vendors can't guarantee.
3. Speed of innovation
One of the main reasons I want to go to the cloud is faster innovation. Since vendors only need to test in their environment, versus every possible combination on companies, they can dedicate more time to new features.
To show me this, you should be able to show regular release schedules and what features are in each one. I understand that not all features are created equal. Quoting me a number of features, is going to make me suspicious without some context. I don't consider changing the columns on a report to be a new feature, or changing the background color.Show me real, new, useful features or I'm going to think you are pulling a fast one.
4. Stop spreading FUD
Many companies, who don't yet have a cloud offering are going to try and confuse people with FUD, Fear Uncertainty and Doubt. They will talk about security breaches or reliability and try to convince them that cloud isn't ready. Sadly at least once a month we get another big story about a cloud outage or security breach so there is no lack of examples.
The reality though is cloud is no more likely to go down than an onsite solution. The big difference is you hear about it a lot more. No one would ever publish an article about a 1000 person company losing email service for 4 hours, but when that happens to Google or Yahoo it's front page news.
Same thing with security. I'm pretty sure a company like Google can hire some pretty darn good security administrators and can afford any solution they want, most companies can't. The downside is Google also gets attacked a lot more than a smaller company. They are a bigger target.
My advice, use reality and facts to educate me. Trying to scare me into not using cloud is just going to make me think you are hiding something.
5. Don't try to sell me 99.999% uptime.
Let's be honest, most people will not be able to tell the difference between a service that is 99.999% available and one that is 99.9% available. One is less than six minutes of downtime a year and the other is slightly under nine hours, which sounds like a big difference, but almost any outage is really going to take longer than 5 minutes to fix. Rebooting a system, which is what most IT people do as a first step can take that long.
I think of this as phone call quality. That used to be a big deal with PBX vendors, and sometimes they will still talk about that, but in the real world, we are all used to cell phone quality, which isn't very good.
While we are being honest, the bigger concern with cloud is the internet connectivity which is going to be the problem more often than not. WAN circuits, DNS servers etc are all lumped under "third party services" which you as a cloud vendor can't guarantee anyway.
Make sure you can explain why your service is available enough but don't make that be your main selling point. I expect it to be in a multiple redundant data center with redundancy and controls in place to avoid as much human error as possible.
Summary: If you really have a cloud product, sell me on the features that really can help me. Don't try to scare me or BS me. Many of us have been working in IT a long time and have really good detection mechanisms. Plus with social media, you might get away with it once or twice but eventually, and usually real soon, people will catch on.
Thursday, September 27, 2012
Monday, September 24, 2012
More contract killers
I spent the weekend reading cloud contracts and was amazed at some of the phrases I saw in some of them. These are the worst ones that I found.
1. We reserve the right to modify or replace these Terms at any time in our sole discretion.
In other words, whatever we agree to we may decide to change our end at any time and with no reason or notice.
2. Company may increase or add new fees for any existing Free or Paid Service or Free or Paid Service feature at any time without notice. Such notice will be posted on Company's Websites. YOU AGREE THAT YOU ARE RESPONSIBLE FOR CHECKING THE COMPANY's WEBSITES EACH MONTH TO CONFIRM WHETHER THERE ARE ANY NEW FEES AND, IF SO, THEIR EFFECTIVE DATE(S).
We agreed to a price but we may decide to change it. You need to ask us every month if the price increased and when we decided to change the price.
3. "We reserve the right, to temporarily suspend or terminate your access to the Service at any time in our sole discretion, with or without cause, and with or without notice, without incurring liability of any kind." .... "You acknowledge that if your access to the Service is suspended or terminated, you may no longer have access to the Content that is stored with the Service."
We might turn you off for no reason and you can't do anything about it. If we do, we don't have to give you any of your stuff back either.
Now imagine if these were contracts for something more tangible, say a rental agreement on your house. In theory, with no notice, you could go home and find your rent had gone up 300%, your door is locked with all the furniture in it and your new apartment is now in New Jersey instead of Manhattan.
I have not heard of anyone getting abused like that, and many times companies will have amendments that probably override the standard terms, so it may not be as bad is it sounds. But the point is this, If you are not reading and understanding these contracts carefully, you may be getting in trouble down the road.
It is possible to get cloud vendors to agree to custom amendments but only if you ask and are willing to walk away. There is a reason that legal is one of the 5 components of a CLOUDscore. Be careful...
Now the legal disclaimer. We are not contract attorneys. If you have questions you should check with one.
1. We reserve the right to modify or replace these Terms at any time in our sole discretion.
In other words, whatever we agree to we may decide to change our end at any time and with no reason or notice.
2. Company may increase or add new fees for any existing Free or Paid Service or Free or Paid Service feature at any time without notice. Such notice will be posted on Company's Websites. YOU AGREE THAT YOU ARE RESPONSIBLE FOR CHECKING THE COMPANY's WEBSITES EACH MONTH TO CONFIRM WHETHER THERE ARE ANY NEW FEES AND, IF SO, THEIR EFFECTIVE DATE(S).
We agreed to a price but we may decide to change it. You need to ask us every month if the price increased and when we decided to change the price.
3. "We reserve the right, to temporarily suspend or terminate your access to the Service at any time in our sole discretion, with or without cause, and with or without notice, without incurring liability of any kind." .... "You acknowledge that if your access to the Service is suspended or terminated, you may no longer have access to the Content that is stored with the Service."
We might turn you off for no reason and you can't do anything about it. If we do, we don't have to give you any of your stuff back either.
Now imagine if these were contracts for something more tangible, say a rental agreement on your house. In theory, with no notice, you could go home and find your rent had gone up 300%, your door is locked with all the furniture in it and your new apartment is now in New Jersey instead of Manhattan.
I have not heard of anyone getting abused like that, and many times companies will have amendments that probably override the standard terms, so it may not be as bad is it sounds. But the point is this, If you are not reading and understanding these contracts carefully, you may be getting in trouble down the road.
It is possible to get cloud vendors to agree to custom amendments but only if you ask and are willing to walk away. There is a reason that legal is one of the 5 components of a CLOUDscore. Be careful...
Now the legal disclaimer. We are not contract attorneys. If you have questions you should check with one.
Thursday, September 20, 2012
SaaS, PaaS and IaaS
Cloud means a lot of different things to different people. Sofwtare, Platform and Infrastructure as a service (SaaS, PaaS, and IaaS for those that like acronyms), are the three main types of cloud today.
How do you know which is best? Well think of it this way, the higher up the stack you go, the less work you need to do to get the benefits of it. Let me explain what I mean.
Salesforce.com is a SaaS company primarily. Their sales cloud is the offering that started the cloud shift. Salesforce.com customers, don't need to write code to leverage the application. In most cases you can use the user interface and pretty easily make new reports, or add new fields to existing forms etc. You get out of having to develop. You also don't need to patch, maintain, backup anything. They handle it all. The trade off is flexibility. You aren't in control of when you get upgraded to the next release, and can't make changes outside of the ones in the user interface.
Amazon web services, or Microsoft Azure are examples of PaaS. They handle all the machine related things, like patches, backups, making sure power is redundant etc. They also give you a great programming interface that lets you easily build new applications. The key here is YOU build the software, but not the operating system. It's a step up from having to make sure you have the latest version of SQL or Linux, but it's more work than using someones else's application. It also gives you more flexibility and control over upgrades of the software.
Infrastructure as a Service puts you back in charge of the operating system but still keeps you from having to worry about details like, redundant hard drives or plugging power supplies into different circuits. You maintain the operating system and have to worry about patches. You need to decide if you want SQL or Postgres for a database. You need to decide on a version of Apache or IIS etc. Then you have to build the application (or buy it of course).
All of these are designed to be scalable (both up and down) and can help avoid a big capital expense up front. The key is to decide how much flexibility you can trade for ease of use.
How do you know which is best? Well think of it this way, the higher up the stack you go, the less work you need to do to get the benefits of it. Let me explain what I mean.
Salesforce.com is a SaaS company primarily. Their sales cloud is the offering that started the cloud shift. Salesforce.com customers, don't need to write code to leverage the application. In most cases you can use the user interface and pretty easily make new reports, or add new fields to existing forms etc. You get out of having to develop. You also don't need to patch, maintain, backup anything. They handle it all. The trade off is flexibility. You aren't in control of when you get upgraded to the next release, and can't make changes outside of the ones in the user interface.
Amazon web services, or Microsoft Azure are examples of PaaS. They handle all the machine related things, like patches, backups, making sure power is redundant etc. They also give you a great programming interface that lets you easily build new applications. The key here is YOU build the software, but not the operating system. It's a step up from having to make sure you have the latest version of SQL or Linux, but it's more work than using someones else's application. It also gives you more flexibility and control over upgrades of the software.
Infrastructure as a Service puts you back in charge of the operating system but still keeps you from having to worry about details like, redundant hard drives or plugging power supplies into different circuits. You maintain the operating system and have to worry about patches. You need to decide if you want SQL or Postgres for a database. You need to decide on a version of Apache or IIS etc. Then you have to build the application (or buy it of course).
All of these are designed to be scalable (both up and down) and can help avoid a big capital expense up front. The key is to decide how much flexibility you can trade for ease of use.
Saturday, September 15, 2012
We all have different cloud needs
One of the things that is important to remember is not all of have the same needs. In fact even in the same company different things will be important depending on the solution you are building.
For example if you are using a cloud based backup, cost is probably important but reliability may not be that important. If the backup doesn't run at 10:00AM, then it will run when the site comes back up. If however you are running your entire sales team on salesforce.com than it absolutely has to be running or you aren't selling.
One of the things we take into account, and recommend for everyone, is to use a weighted matrix that takes into account that needs change. When we calculate a CLOUD score we take this into account and using out algorithm convert everyone to a 0-100 score, even though the needs change.
Eventually we will add the ability to use other's rankings, with your weightings to better leverage what others think. It's important to get as much data as possible and then use it properly to make sure you get the right solution.
For example if you are using a cloud based backup, cost is probably important but reliability may not be that important. If the backup doesn't run at 10:00AM, then it will run when the site comes back up. If however you are running your entire sales team on salesforce.com than it absolutely has to be running or you aren't selling.
One of the things we take into account, and recommend for everyone, is to use a weighted matrix that takes into account that needs change. When we calculate a CLOUD score we take this into account and using out algorithm convert everyone to a 0-100 score, even though the needs change.
Eventually we will add the ability to use other's rankings, with your weightings to better leverage what others think. It's important to get as much data as possible and then use it properly to make sure you get the right solution.
Sunday, September 9, 2012
Cloud and Internet redundancy
In our last post we talked about contracts and SLA's and while they are important, not all of the problems are actually the cloud provider's responsibility. Some of the problem could be your network.
Historically internet access wasn't always considered mission critical, but when your financial system, customer support, sales and storefront are online, it needs to be. Not all companies have figured that out yet.
The best redundancy is dual carrier. Some carriers will sell you a redundant option and this is good, but doesn't always protect you from "logic" errors. Internet routing is complex and like any complex system can have problems. These problems can sometimes, and frankly pretty rarely, but sometimes affect the provider network, even if you have redundant physical connections.
Other times the problem could be with the ISP's uplinks and who they interconnect with. This isn't really a problem with the bigger network players since they have multiple uplinks to all the other providers, but smaller regional providers could have limited redundancy of their own. Also do not forget your own redundancy. If both circuits go into the same closet and a water pipe breaks and ruins both sets of equipment that is just as bad.
It's also possible, though not a technical problem, that something silly like a billing problem could cause the provider to terminate your service, either intentionally or not. I have seen cases where a $6.24 bill that was billed incorrectly, and sent to the wrong address, caused the provider to terminate services. Getting circuits turned back on, for some reason, is a lot more difficult than turning them off.
So two carriers is best, but even then many providers resell other providers services and even using two carriers isn't foolproof. You have to ask which paths the circuits take, where they locally terminate and then how they get to a major POP. I have seen cases where two providers were both using the same fiber and a single fiber cut took down both providers and both of our redundant circuits.
Even using a wireless provider for a backup isn't foolproof. In Lawrence MA a few weeks ago a mattress fire took down a major fiber and copper conduit which took many cellular providers offline as well as the phones and internet for many local businesses. It even took down emergency 911 service, and if anyone takes redundancy seriously it's those guys. Lives really are at stake if 911 isn't working. Luckily they had coverage with other towns to assist.
Getting internet redundancy is hard and not as foolproof as we would like. Make sure to ask questions around where they circuit goes, who the interconnect with and what the escalation points are in case something does go wrong. Redundancy is difficult to setup but not impossible. As more services move to the cloud, it is important to make sure you can always get there.
Historically internet access wasn't always considered mission critical, but when your financial system, customer support, sales and storefront are online, it needs to be. Not all companies have figured that out yet.
The best redundancy is dual carrier. Some carriers will sell you a redundant option and this is good, but doesn't always protect you from "logic" errors. Internet routing is complex and like any complex system can have problems. These problems can sometimes, and frankly pretty rarely, but sometimes affect the provider network, even if you have redundant physical connections.
Other times the problem could be with the ISP's uplinks and who they interconnect with. This isn't really a problem with the bigger network players since they have multiple uplinks to all the other providers, but smaller regional providers could have limited redundancy of their own. Also do not forget your own redundancy. If both circuits go into the same closet and a water pipe breaks and ruins both sets of equipment that is just as bad.
It's also possible, though not a technical problem, that something silly like a billing problem could cause the provider to terminate your service, either intentionally or not. I have seen cases where a $6.24 bill that was billed incorrectly, and sent to the wrong address, caused the provider to terminate services. Getting circuits turned back on, for some reason, is a lot more difficult than turning them off.
So two carriers is best, but even then many providers resell other providers services and even using two carriers isn't foolproof. You have to ask which paths the circuits take, where they locally terminate and then how they get to a major POP. I have seen cases where two providers were both using the same fiber and a single fiber cut took down both providers and both of our redundant circuits.
Even using a wireless provider for a backup isn't foolproof. In Lawrence MA a few weeks ago a mattress fire took down a major fiber and copper conduit which took many cellular providers offline as well as the phones and internet for many local businesses. It even took down emergency 911 service, and if anyone takes redundancy seriously it's those guys. Lives really are at stake if 911 isn't working. Luckily they had coverage with other towns to assist.
Getting internet redundancy is hard and not as foolproof as we would like. Make sure to ask questions around where they circuit goes, who the interconnect with and what the escalation points are in case something does go wrong. Redundancy is difficult to setup but not impossible. As more services move to the cloud, it is important to make sure you can always get there.
Wednesday, September 5, 2012
Cloud contracts
As we started scoring cloud vendors, the biggest area that pretty much all the vendors do poorly at is contracts.
Some of the questions we ask about contracts are:
If I want to leave, for example if they get bought by a competitor, can I get my data out and go?
This varies, but often times the ability to leave is either not mentioned, or it explicity says "You can leave when the term is up".
Many times having the data by itself is almost useless, unless you have someplace to put it. Migrating out of salesforce.com without being able to re-use that data in another system would be difficult and costly.
Are there financial penalties for service level agreement failure.
These are almost universally, a joke. If you read closely most of them say "If the service is unavailable for a day, we will refund up to 3% of the monthly fee". This pretty much equates to, we won't charge you if you didn't get to use it. Not much incentive to fix a problem.
Other times we have seen clauses that say "You will receive this credit, but you must notify us, in writing, within 72 hours". Unless you are really good, that never happens.
Are maximum increases baked in?
By default this never happens, mostly because the vendors don't want you to think about that. In essence though the cloud vendors, once they hook you, can increase the rates until it is almost worth going somewhere else. Sort of like gas prices in the US. They, whoever they are, raise the rates until it's almost worth buying a more fuel efficient vehicle, or driving less. Almost, but not quite.
How much notice do you need to give us to terminate?
Rarely, but it has happened, a vendor will terminate you. If that happens how much notice do they need to give you? Is that enough time to find a new vendor, migrate the data and train on the new system? Usually it's either 30 or 60 days. You realistically probably need 6 months for a big SaaS. If it's something more commodity like VMware in the cloud, 30 days may be fine.
How much notice do we need to give them if we want to leave?
As a customer you want this to be as little as possible. Normal is 30 days. We have seen some that require 6 months notice.
Does the contract auto-renew? If so what are the terms?
Ideally this auto-renews at the current rate, but sometimes it will cancel, which if you are not expecting that, can be horrible. Other times it will renew, but at list price. If, for example, you negotiated 50% off of list, that means your monthly bill just doubled. Even worse, it may have doubled for the next 12 or 24 months depending on the contract.
Cloud contracts can be tricky and many vendors aren't used to amending their terms. Only by working together can we encourage vendors to be a little better about what we are allowed to do as customers.
Some of the questions we ask about contracts are:
If I want to leave, for example if they get bought by a competitor, can I get my data out and go?
This varies, but often times the ability to leave is either not mentioned, or it explicity says "You can leave when the term is up".
Many times having the data by itself is almost useless, unless you have someplace to put it. Migrating out of salesforce.com without being able to re-use that data in another system would be difficult and costly.
Are there financial penalties for service level agreement failure.
These are almost universally, a joke. If you read closely most of them say "If the service is unavailable for a day, we will refund up to 3% of the monthly fee". This pretty much equates to, we won't charge you if you didn't get to use it. Not much incentive to fix a problem.
Other times we have seen clauses that say "You will receive this credit, but you must notify us, in writing, within 72 hours". Unless you are really good, that never happens.
Are maximum increases baked in?
By default this never happens, mostly because the vendors don't want you to think about that. In essence though the cloud vendors, once they hook you, can increase the rates until it is almost worth going somewhere else. Sort of like gas prices in the US. They, whoever they are, raise the rates until it's almost worth buying a more fuel efficient vehicle, or driving less. Almost, but not quite.
How much notice do you need to give us to terminate?
Rarely, but it has happened, a vendor will terminate you. If that happens how much notice do they need to give you? Is that enough time to find a new vendor, migrate the data and train on the new system? Usually it's either 30 or 60 days. You realistically probably need 6 months for a big SaaS. If it's something more commodity like VMware in the cloud, 30 days may be fine.
How much notice do we need to give them if we want to leave?
As a customer you want this to be as little as possible. Normal is 30 days. We have seen some that require 6 months notice.
Does the contract auto-renew? If so what are the terms?
Ideally this auto-renews at the current rate, but sometimes it will cancel, which if you are not expecting that, can be horrible. Other times it will renew, but at list price. If, for example, you negotiated 50% off of list, that means your monthly bill just doubled. Even worse, it may have doubled for the next 12 or 24 months depending on the contract.
Cloud contracts can be tricky and many vendors aren't used to amending their terms. Only by working together can we encourage vendors to be a little better about what we are allowed to do as customers.
Saturday, September 1, 2012
Questions to ask cloud vendors
Part of our methodology is to make sure we ask enough questions to fill out the weighted matrix we use for scoring cloud vendors. When we schedule a vendor interview, we will send them the below questions, not so that they can answer and send them back, but so they can have the right people available on the phone to discuss them. Many times this can prompt other discussions to help us better understand how they operate which helps us better score them
Feel free to use the below in your own cloud scoring. Or if you are too busy and want us to do it, by all means let us know.
--------------------------
We are reviewing our cloud vendors and we would like to have a discussion to understand your company, processes, contracts etc. We are very pro-cloud so please do not be alarmed. We are not using this as a reason to disqualify you, but simply to understand what our risks are and if we need to take additional steps to mitigate these risks. In essence this is our due diligence.
You do not need to answer these questions. We think there is more value in having a discussion around these topucs to we can drill in where we need to.
Disaster Recovery and Business Continuity
Do you have redundant sites designed for auto-failover? How long does it take for the redundant site to take over. Does this include the time to decide to fail over?
What kind of RTO/RPO are in place and are they actually tested against?
Do you have geographic redundancy?
Do you perform backups. and how granular and far back can we recover? Can we recover accidentally deleted files?
What impact does a failed HD, server, cabinet, switch, data center have?
E-discovery
Is it possible to search for any files, records or emails by keyword, owner, date?
Can we do legal holds by user, file, keyword?
Can we get access to “access logs” in the event we need to? If so how far back can we get? What does it show us?
Can you see who your users are sharing with? If so can you easily remove access from an enterprise level?
Are there any additional costs for this?
Stability
Do you have a public, or better yet third party, site that shows real time status for transparent operations?
Is code/data in escrow? If so how often does it get updated?
Can we request a backup or better yet, just take a backup, of our data including any customizations?
Company financials
Can you share financial data for the last few years?
Are you private or publicly owned?
Are you cash flow positive?
Are you adding new customers? How many?
Do you know your NPS (Net Promoter Score)?
Who are the key investors and management team?
Authentication
Do you support automatic provisioning and de-provisioning of user accounts?
Do you support LDAP.RADIUS or even better SAML authentication and authorization back to us?
Do you use encryption? If so is it for data in flight (moving across the network), at rest (on disk or in memory) or both? What kind of encryption is it?
Compliance and Privacy
Do we get notified of an investigation?
Can our data be seized as part of another companies investigation?
Is data recoverable by your organization?
Contract
If I want to leave, for example if they get bought by a competitor, can I get my data out and go?
Are there financial penalties for service level agreement failure.
Are maximum increases baked in?
How much notice do you need to give us to terminate?
How much notice do we need to give them if we want to leave?
Does the contract auto-renew? If so what are the terms?
Performance
Are you globally load balanced?
Do you use a specialized network or other Wan optimization, for better performance?.
Who do you use for WAN connectivity? I
Do you offer “offline” ability? If so it is automatic, or does the user need to know that they will be offline and plan accordingly?
How scalable is the system? Can we scale to 10X, 100X, 1000X? What is the breaking point? And how quickly can new capacity be added if needed to scale higher?
Development
Do you offer built in integration tools? If so to what other systems?
What toolset is used for “custom development”?
Can we customize the login or "main" screens with our logo, colors, etc? How much canned vendor wording needs to stay, or can we wipe it all clean if we want?
How vibrant is your customer community? Please point us to the public discussion Q&A boards for your product. What are the main Twitter hashtags your customers use when they talk about you? Do you monitor them?
Do you gather product ideas publicly from your customers, where they can read and vote on each others' ideas?
Are the API’s well documented? How often do they change?
How much of the system can be operated with API’s versus the user interface?
How is authentication handled in the APIs?
Support
Are you staffed 24/7?
Is it onsite, email, phone, web or all?
What sort of response time is available?
When we open a case, do wetalk to someone knowledgeable right away, or do we have to have a number assigned and wait for someone to call us back?
What is the average tenure of the tier1 staff?
Is there a public knowledgebase available? Is it the same as the internal one or is it filtered? How good is the search or UI?
Can anyone from our company or IT staff call, or do we only get a certain amount of “authorized users”?
Can I use support as part of my evaluation period?
Does the same engineer own a case throughout its lifecycle, or are cases "handed off" among tiers?
Does your support staff prefer or default to email, phone, or web?
What's your support team's onshore / offshore and employee / contractor breakdown?
Does your support team also support your own internal implementation of your software?
Can we log tickets on a portal and carry on the whole conversation there?
Architecture
How quickly do new features show up? Do we need to do anything or do we “magically” get them? How much notice do you give us for training users?
Are you built on a multitenant system?
Do you support multiple clients, like iphone, android, blackberry as well as tablets? Are you strictly HTML5/browser based? If so which browsers?
How long does it take to bring a development or evaluation system online? Does it need human intervention?
Are there any special restrictions on sandbox or developer systems? (storage, API’s, queries, users etc)
Do we have the ability to give customers "sandbox" instances, i.e. exact copies (including metadata and data) of our production instance, for testing purposes? If so, can we create them directly, or does it take a request and lead time? How often can sandboxes be refreshed?
To what extent is the system "metadata-driven," i.e. how much can I change without code?
User Experience
How accurate is the development roadmap for the next release? Next year’s worth of releases?
Are there mobile versions of the application? If so what limitations, if any, are there?
Is user training available online?
How extensive is the help system?
Administration
Can my system administrators "log in as" other users, in order to diagnose problems or confirm security settings?
For an implementation our size, how much IT time is typically required to manage the system?
How detailed are users' activities logged? What can a sysadmin see? e.g. can we see who logged in when; what functions they used; what pages they viewed; etc.
Feel free to use the below in your own cloud scoring. Or if you are too busy and want us to do it, by all means let us know.
--------------------------
We are reviewing our cloud vendors and we would like to have a discussion to understand your company, processes, contracts etc. We are very pro-cloud so please do not be alarmed. We are not using this as a reason to disqualify you, but simply to understand what our risks are and if we need to take additional steps to mitigate these risks. In essence this is our due diligence.
You do not need to answer these questions. We think there is more value in having a discussion around these topucs to we can drill in where we need to.
Disaster Recovery and Business Continuity
Do you have redundant sites designed for auto-failover? How long does it take for the redundant site to take over. Does this include the time to decide to fail over?
What kind of RTO/RPO are in place and are they actually tested against?
Do you have geographic redundancy?
Do you perform backups. and how granular and far back can we recover? Can we recover accidentally deleted files?
What impact does a failed HD, server, cabinet, switch, data center have?
E-discovery
Is it possible to search for any files, records or emails by keyword, owner, date?
Can we do legal holds by user, file, keyword?
Can we get access to “access logs” in the event we need to? If so how far back can we get? What does it show us?
Can you see who your users are sharing with? If so can you easily remove access from an enterprise level?
Are there any additional costs for this?
Stability
Do you have a public, or better yet third party, site that shows real time status for transparent operations?
Is code/data in escrow? If so how often does it get updated?
Can we request a backup or better yet, just take a backup, of our data including any customizations?
Company financials
Can you share financial data for the last few years?
Are you private or publicly owned?
Are you cash flow positive?
Are you adding new customers? How many?
Do you know your NPS (Net Promoter Score)?
Who are the key investors and management team?
Authentication
Do you support automatic provisioning and de-provisioning of user accounts?
Do you support LDAP.RADIUS or even better SAML authentication and authorization back to us?
Do you use encryption? If so is it for data in flight (moving across the network), at rest (on disk or in memory) or both? What kind of encryption is it?
Compliance and Privacy
Do we get notified of an investigation?
Can our data be seized as part of another companies investigation?
Is data recoverable by your organization?
Contract
If I want to leave, for example if they get bought by a competitor, can I get my data out and go?
Are there financial penalties for service level agreement failure.
Are maximum increases baked in?
How much notice do you need to give us to terminate?
How much notice do we need to give them if we want to leave?
Does the contract auto-renew? If so what are the terms?
Performance
Are you globally load balanced?
Do you use a specialized network or other Wan optimization, for better performance?.
Who do you use for WAN connectivity? I
Do you offer “offline” ability? If so it is automatic, or does the user need to know that they will be offline and plan accordingly?
How scalable is the system? Can we scale to 10X, 100X, 1000X? What is the breaking point? And how quickly can new capacity be added if needed to scale higher?
Development
Do you offer built in integration tools? If so to what other systems?
What toolset is used for “custom development”?
Can we customize the login or "main" screens with our logo, colors, etc? How much canned vendor wording needs to stay, or can we wipe it all clean if we want?
How vibrant is your customer community? Please point us to the public discussion Q&A boards for your product. What are the main Twitter hashtags your customers use when they talk about you? Do you monitor them?
Do you gather product ideas publicly from your customers, where they can read and vote on each others' ideas?
Are the API’s well documented? How often do they change?
How much of the system can be operated with API’s versus the user interface?
How is authentication handled in the APIs?
Support
Are you staffed 24/7?
Is it onsite, email, phone, web or all?
What sort of response time is available?
When we open a case, do wetalk to someone knowledgeable right away, or do we have to have a number assigned and wait for someone to call us back?
What is the average tenure of the tier1 staff?
Is there a public knowledgebase available? Is it the same as the internal one or is it filtered? How good is the search or UI?
Can anyone from our company or IT staff call, or do we only get a certain amount of “authorized users”?
Can I use support as part of my evaluation period?
Does the same engineer own a case throughout its lifecycle, or are cases "handed off" among tiers?
Does your support staff prefer or default to email, phone, or web?
What's your support team's onshore / offshore and employee / contractor breakdown?
Does your support team also support your own internal implementation of your software?
Can we log tickets on a portal and carry on the whole conversation there?
Architecture
How quickly do new features show up? Do we need to do anything or do we “magically” get them? How much notice do you give us for training users?
Are you built on a multitenant system?
Do you support multiple clients, like iphone, android, blackberry as well as tablets? Are you strictly HTML5/browser based? If so which browsers?
How long does it take to bring a development or evaluation system online? Does it need human intervention?
Are there any special restrictions on sandbox or developer systems? (storage, API’s, queries, users etc)
Do we have the ability to give customers "sandbox" instances, i.e. exact copies (including metadata and data) of our production instance, for testing purposes? If so, can we create them directly, or does it take a request and lead time? How often can sandboxes be refreshed?
To what extent is the system "metadata-driven," i.e. how much can I change without code?
User Experience
How accurate is the development roadmap for the next release? Next year’s worth of releases?
Are there mobile versions of the application? If so what limitations, if any, are there?
Is user training available online?
How extensive is the help system?
Administration
Can my system administrators "log in as" other users, in order to diagnose problems or confirm security settings?
For an implementation our size, how much IT time is typically required to manage the system?
How detailed are users' activities logged? What can a sysadmin see? e.g. can we see who logged in when; what functions they used; what pages they viewed; etc.
Not all clouds are created equal
One of the great things with making cloud software is it is really easy to develop incredible useful and scalable software. Unfortunately there is a downside to that too.
Years ago, when you wanted to start a company you had to spend a lot of money and time to get it up and running. That meant you had to be very dedicated to it, since you typically had your life savings, and in many cases your friends and family's money too.
With companies so easy to start, it's also real easy to shut them down and many cloud companies have been closed down without much effort. That is one of the risks to cloud companies. It's important to understand how invested the management team is before you start betting your company on them.
Another challenge with "cloud vendors" is since cloud has become the hot phrase of 2012, everyone is suddenly a cloud vendor. Companies that have been selling products for years suddenly put cloud in front of it and try to pretend it is something different.
True cloud vendors are built from the ground up differently. They are built to take advantage of the benefits of multi-tenancy. They are built so scale seamlessly. They don't require new client software to be installed.
It's important to understand the architecture to be able to weed out the false cloud vendors from the real ones.
This blog will help explain how to decide which cloud vendors are good and which ones are trying to spin a tale of innovation to cover their lack of it. Hopefully it will help you. We would love it if you help share what you learn at www.sharethecloud.com and let others learn from your experiences.
Years ago, when you wanted to start a company you had to spend a lot of money and time to get it up and running. That meant you had to be very dedicated to it, since you typically had your life savings, and in many cases your friends and family's money too.
With companies so easy to start, it's also real easy to shut them down and many cloud companies have been closed down without much effort. That is one of the risks to cloud companies. It's important to understand how invested the management team is before you start betting your company on them.
Another challenge with "cloud vendors" is since cloud has become the hot phrase of 2012, everyone is suddenly a cloud vendor. Companies that have been selling products for years suddenly put cloud in front of it and try to pretend it is something different.
True cloud vendors are built from the ground up differently. They are built to take advantage of the benefits of multi-tenancy. They are built so scale seamlessly. They don't require new client software to be installed.
It's important to understand the architecture to be able to weed out the false cloud vendors from the real ones.
This blog will help explain how to decide which cloud vendors are good and which ones are trying to spin a tale of innovation to cover their lack of it. Hopefully it will help you. We would love it if you help share what you learn at www.sharethecloud.com and let others learn from your experiences.
Subscribe to:
Posts (Atom)