I had a few questions in regards to elasticsearch architecture and associated services and/or products that is not clear to me.
The idea is to setup an elasticsearch instance for searching through file shares, Exchange mailboxes, Sharepoint sites and even Teams conversations if possible.
How would I setup the elasticsearch instance to support the following requirements:
Security filtering results from these sources for users
Develop on a simple and clean web search page like SearchUI from Elastic themselves.
Active Directory or ADFS authentication
Use nodejs on a separate server to proxy to elastic, as elastic user management means that users get access to all search results
I can find tutorials and blogs on some of these items, but no comprehensive description of how the architecture would actually work specifically with the SearchUI and proxying of data to ES.
Please have a look at this new product released by Elastic guys using same elastic search framework
https://www.elastic.co/workplace-search
it closley matches your requirement.
I am looking for a frontend to query Elasticsearch, I know that Kibana is the best option for this (Besides grafana).
Unfortunately Kibana is totally not mobile friendly on android. Search is a real headache...
Tried searching on google but not able to find a worthy alternative.
Someone has experience on other tools/making kibana perform better on mobile devices.
If this turns out to be nothing I am seriously considering writing an angular app to perform base searches myself. Delivering in the dynamicniss of Kibana itself :(
It is painful to use web version of kibana on ipad, although they claim it is mobile friendly.
I see a recent post on elastic forum, that a native iphone/ipad kibana go is now available, a very nice start.
https://discuss.elastic.co/t/ios-mobile-app/218463
You can download the app and check out the demo dashboard, to connect to your own ELK, you will need to install a kibana plugin then pointing to your own kibana.
I'm a hobby developer that use Parse.com as my database and website host, as Facebook is going to shutdown Parse.com, I'm now looking for alternative Parse server.
I use Parse's Cloud Code Hosting to build Dynamic Web App, and Parse itself to store data collected from the website I've build, with custom Cloud Code to help getting and managing data. I've build a Windows Phone app for myself to manage the data I've collected too.
Is there any alternative server that has my requirement?
Dynamic Website
Database host
Custom Cloud Code (with BeforeSave and AfterSave trigger)
with Windows Phone SDK (or REST API if doesn't have)
Very thank you for helping me!
Try out Hasura.
Hasura (http://www.hasura.io): Hasura is a neat PaaS + BaaS solution. It is now competing with Firebase, Kinvey, Heroku et al. There is a full comparison page here: Compare | Hasura (https://compare.beta.hasura.io) . The difference majorly lies in infra ownership as well as no tech lock-in due to open-source components(like docker, kubernetes,postgres) building the major chunk of the platform. Check it out. There is also an option to explore (https://explore.beta.hasura.io/) Hasura by building your own blog web app and a todo app in under 15 mins.
Hasura should fit in perfectly for your needs.
DISCLAIMER : Hasura Engineer here.
I'm using Simbla website application development. It doesn't support all of your requirments but it has great UI builder with a backend parse database.
You can try using the parse open source server it has cloud code and you can use a custom database with it.
I have a project and I'm planning to start the web app as an Azure Web Site and then migrate it to an Azure Cloud Service (also called Hosted Service) if it is needed as a scale strategy.
The decision is because I read that Azure Web Sites are more simple and fast to develop with almost no Azure-specific configurations or code. So starting fast and simple is a good starting point for the project.
But, is that a good starting point for you?
Is migrating an Azure Web Site to an Azure Cloud Service the same as you were migrating a normal ASP.NET Website to an Azure Cloud Service?
Would you start with an Azure Cloud Service right from the beginning? If yes, why?
Thanks for your time.
There are benefits to both deployment models, it will eventually come down to what you are trying to achieve and ultimately the success of your application.
Below I've outlined the Pros and Cons of each of the models to ensure that you're making the right choice for your applications goals.
Windows Azure Web Sites
You have properly identified that Windows Azure Web Sites is a great starting point for an application, however you could also consider that Web Sites does offer enough scalability for many solutions.
Pros
10 Free sites during preview [Free for 12 months]
Easy Deployment (use Git, TFS, Web Deploy or FTP)
Quick Scalability (You can move to your own dedicated cluster [aka reserved standard])
Simple Development (Supports Classic ASP, ASP.NET, Node.js, Python & PHP)
Persistent Environment (most people are used to this)
Cons
No SSL Support on Custom Domains
in Preview (currently no SLA)
Windows Azure Cloud Services
Cloud Services (formerly known as Hosted Services) is definitely the vision for the future of Web Applications. It is built with resiliency in mind to keep the cost of applications affordable by scaling to meet demand, and dial back capacity when your traffic slows.
Pros
Increased control over the cost of your application (if architected correctly)
Flexibility (You have full control over the environment)
SSL Support
Language Agnostic
Web Server Agnostic (although IIS is available by default)
Auto Management of Servers
Cons
Architecture should be carefully considered
Deployment time is slower (Slows development cycle)
Things to consider for Portability
The items above might have given you enough to plan the immediate future of the application and it is very likely that you might want to consider Cloud Services in the future (it fits a number of application scenarios better in the long run).
Here is a list of things to help portability between Web Sites to Cloud Services:
Start thinking Stateless
Windows Azure Web Sites is nice as it is a persistent environment, which means you are able to store things like session state and assets to the disk.
Although this is a good feature, it's best to start planning towards a stateless application, if your end goal is to be in Cloud Services. Here are a few things you can do to start thinking stateless:
Don't rely on Session State
If you need it, come up with a strategy to make it scale (Caching Service, SQL, or Storage)
Use the Storage Service
Assets such as Static HTML, css, javascript and images are better placed in Storage
Avoids additional bandwidth on your Web Site (potentially stay shared longer for lower cost)
Can be CDN Enabled, provides a better experience for International markets
Easier to update web assets when application is migrated to Cloud Services
Storing User content
If your application already stores to the Storage Service, there is one less code modification in the future when moving to cloud services.
Make it easy to discover patterns in your Data
The benefit of Cloud Services is it enables you to reduce cost by only scaling what needs scaled. Starting the process of identifying your scale units i.e. How you partition your database or Tables in Storage.
I read all post and all of them are very helpful.
In addition to all post , I found an info on msdn : Windows Azure Websites, Cloud Services, and VMs: When to use which?
With Windows Azure Websites you can:
Build highly scalable web sites on Windows Azure.
Quickly and easily deploy sites to a highly scalable cloud environment that allows you to start small and scale as needed.
Use the languages and open source applications of your choice then deploy with FTP, Git or TFS, and easily integrate Windows Azure services like SQL Database, Caching, CDN and Storage.
With Cloud Services you can:
Build or extend your enterprise applications on Windows Azure.
Create highly-available, scalable applications and services using a rich PaaS environment. Support advanced multi-tier scenarios, automated deployments and elastic scale. Deliver great SaaS solutions to customers anywhere around the world.
And also there is summarizes the option on msdn :
And comparing some features Web Sites and Cloud Services on msdn:
Azure is a great place to have your app, but there are some considerations you need to know before start migrating it.
Azure Websites and Hosted Services are really trivial to deploy. With
Visual studio you generate the package and simply upload it. Then you
have a Development environment to check it. If it's ok for you, swap
ips. If it's not ok for you, upgrade again.
Your instances have some properties that could be annoying. For
example, you cannot be sure about your IP. Then if your app works
with some provider using IP restriction, you will need to figure out
how to proceed.
More considerations. Your "server" could be reimaged at any moment.
If you store something on the local disc, that file could go away at any moment.
Azure works very nice if you have at least 2 instances or more for
each website. Maybe your app is not prepared for that. The first step
will be managing the sessions with the appFabric. Is really
easy, just a change on your web config. Be careful because this
session state doesn't work exactly as the "old one". You cannot store
non-serializable objects (should be easy to adapt) or a very large objects (more than 8MB).
If you are going to develop something from zero, I suggest you to start into azure from the beginning. The reason is simple: it's really cheap to start and you will not pay serious money until the app have lot's of visits. It's also very cheap to setup a SQLAzure and a storage account. One you have all in place, it's easy to add more instances or scale up.
Example:
Imagine you have an idea and you wish to show up to some possible investors.
You start setting up a little SQLAzure database (1GB ), $9,99 monthly.
Then you build a site and you put 2 extra small instances, $18,72 monthly.
Let's say you need 100 GB of space (images, backups, ...), $12,50 monthly.
At his point, you have all in place to start your business paying less than $50 monthly.
If you site have exit and the visits starts to come, you change your instances for small instances (it's really dangerous to have production environment with extra small instances, because do not have cpu reservation). Then you change the extra small cost ($18,71) up to $57,60. Maybe you need more space to that SQL Azure? etc...
prices calculated from here: http://www.windowsazure.com/en-us/pricing/calculator/?scenario=web .
Those are few tips, there is a lot more. My advice is to start a trial account and play with it.
Final advice: Its very easy to solve everything just purchasing more resources. Sometimes you need to refactor and optimize your code. If you simply add more resources each time you have a problem, you could end with a huge bill and a very messy code.
Hope it helps!
Another advantage of Windows Azure Cloud Services over Web Sites is that a cloud service can be added to an Azure Virtual Network. This can give it access to on-premises resources like databases. So if your requirements are such that you need the scalability offered by Azure but need to keep your data on-premises due to security restrictions, cloud services is a better choice.
Azure web sites cannot be part of an Azure virtual network. To access on-premises resources mechanisms such as Azure Service Bus Relay must be configured.
We've had our web site running on PHP on some hosting and at some point decided to move it to Azure (where sits main part of our service). We've started with Azure Web Sites which was great from development point of view (mainly integration with git). But after about a week of testing (when we've decided to actually move the production web site) we've found that currently
No SSL for custom domains
Custom domains are available only for reserved instances (no shared infrastructure)
SLA
So we moved to Hosted Service. The main problem for us was lack of ability of simple deployment (need to build package and upload whole package of the web site), and found solution was to use dropbox - as a startup task for role, we're installing dropbox service on the machine, which takes all the web site from dropbox, which in turn have SVN checked out folder, so site updates became very easy.
I heard that apps don’t port directly and they have to be specifically written to work with Azure. I’m new to Azure and did some reading but I don’t see anything from their site or documentation that suggests that you must specifically code for Azure, so is it true?
If this question is better suited for another StackExchange site please let me know.
You should take a look at the Windows Azure Platform Training Kit to see some introductory project labs.
While you theoretically can just "drop in" some apps into a Windows Azure VM, it hardly makes sense to do so. Some apps can be migrated with only minor changes, such as an asp.net website - session state is easily handled by AppFabric Cache, and the Web Role VM is Windows Server 2008 with Full IIS. However, you'd still need to integrate with Windows Azure Diagnostics, to be able to have external visibility to the health of your app.
Further, with the example above, there are more optimal things you can do, such as moving static content to blob storage, and taking load off your VMs. This requires you to upload this content, and then change your IMG and other tag URLs to point to a slightly-different URL.
Just remember that you're moving to Windows Azure to take advantage of its platform and related services, not just to be a rack to host a server. To take advantages of these services, you're going to have to target them specifically, such as:
Access control services
Caching
Connectivity (vpn, service bus)
Diagnostics
Database (SQL Azure)
Synchronization services
Traffic management across data centers
This is somewhat true. Your apps will typically run as-is. You just need to add an azure project to the solution, and add your MVC apps as roles to the Azure Project.
I found O'Reilly's "Programming Windows Azure" to be a good conceptual introduction to Azure.