What is MONGO DB?

MongoDB is a cross-platform document based system. Classified as a "NoSQL"database, MongoDB eschews the traditional relational database structure in favor of JSON-like documents with dynamic schemas , making the integration of data in certain types of applications easier and faster.

It is very interesting to work on a system which does not follow the conventional tantrums of relational DB, and curbing the obligation of maintaining  a perfect table structure and keeping the relations intact. Now you need not worry for the same with MongoDB , you just have to create the collections to much of a surprise the collections here are dimensionless , what I mean is that there could be a scenario where in the first row you have 3 records and next you have 2, wherein the SQL table you need to specify all the fields for all the rows. For Instance :
As the data stored is in a form of a document so all the documents are like a JSON  Document
{a:1,b:1,c:3},
{a:1},
{a:2,b:3},
{a:1,b:4,c:3,d:4}

As you could see above , its dimensionless. Now the next question which would haunt you would be that how would the sorting, grouping and other aggregations can be done , so the answer is that MongoDB has an inbuilt feature which skips the rows which does not contain the queried field to sort.

One might question that if there is no relational arrangement then how come the relations are maintained, but joy of a Document database is that it eliminates lots of Joins. Your first instinct should be to place as much in a single document as you can. Because MongoDB documents have structure, and because you can efficiently query within that structure there is no immediate need to normalize data like you would in SQL. In particular any data that is not useful apart from its parent document should be part of the same document.So the catch is you can put as much data in the parent document which has high frequency of access , but is required to be updated less . And the documents which needs lots of updates or require increments it could be placed in a small document. So in that way the data is arranged and that data is only accessed which needs to be retrieved.


'mongod' This is a command line command to run the database instance , 'mongo' is a command to run the mongo shell where in you could access you data ,'mongos' this command keeps the track of shard keys. When you have to access the data you have to run the database instance and then access the mongo shell .

MongoDB provides a number of features that allow application developers and database administrators to customize the behavior of a sharded cluster or replica set deployment so that MongoDB may be more “data center aware,” or allow operational and location-based separation.

MongoDB uses write ahead logging to an on-disk journal to guarantee write operation durability and to
provide crash resiliency. Before applying a change to the data files, MongoDB writes the change operation to the
journal. If MongoDB should terminate or encounter an error before it can write the changes from the journal to the 
data files, MongoDB can re-apply the write operation and maintain a consistent state.

The MongoDB aggregation framework provides a means to calculate aggregated values without having to use mapreduce.While map-reduce is powerful, it is often more difficult than necessary for many simple aggregation tasks,such as totaling or averaging field values.
If you’re familiar with SQL, the aggregation framework provides similar functionality to GROUP BY and related
SQL operators as well as simple forms of “self joins.” Additionally, the aggregation framework provides projection
capabilities to reshape the returned data. Using the projections in the aggregation framework, you can add computed fields, create new virtual sub-objects, and extract sub-fields into the top-level of results.

MongoDB supports unique replication feature which increases data availability with multiple copies of data on different database servers, replication protects a database from the loss of a single server. Replication also allows you to recover from hardware failure and service interruptions. With additional copies of the data, you can dedicate one to disaster recovery, reporting, or backup

This feature gives and edge in the case when the primary DB fails, because mongoDB supports a unique feature , wherein it grants  the secondary databases to perform an election amongst them to elect the primary database and then carry on the further transactions from that instance.


Another great feature of MongoDB is Sharding, or horizontal scaling which divides the data set and distributes the data over multiple servers, or shards. Each shard is an independent database, and collectively, the shards make up a single logical database. 
MongoDB also supports segregation based on functional parameters, to ensure that certain mongod instances are only used for reporting workloads or that certain high-frequency portions of a sharded collection only exist on specific shards. 

There is a limitation of MongoDB that it can have a document of maximum 16mb ,not greater than that , but to cater that 'GridFS' is there- This breaks the document in several small chunks and index it accordingly . This proves handy when you have to store huge data like photos,etc.

In my views it really fascinating to work on MongoDB ,to experience the NOSQL environment , it really great to provide flexibility in the database structure as the structure, relations have been the prime concern  for the developers . I would certainly recommend the usage in order to provide wings to the application.

Well one of the courses which I would recommend everyone is MongoDB for Developers M101J-   MongoDB for Developers M101J. In the course nicely all the features are covered and explained , I will recommend everyone to complete this.

Don't hesitate to add comments and questions if you have any doubts in the content , I will try to answer as soon as possible.
Cheers!!

Poverty,Poverty,Poverty ... Is this what hindering growth of our Nation!!

   Poverty , when we hear this word many of us give it a thought (that's really great!!) but next thing which they do is just let it go having reason , that we are not poor ,we don't need to think about them etc. etc. As we all are rational beings and we have solid reasons to not to give  thought to grave problem of poverty.

    Million Dollar question is , IS POVERTY HAMPERING THE GROWTH OF NATION??
Truthfully if I say not really because the poor people right now do not constitute the growth of  nation, as they are already out of the race . These people are not even capable of managing their basic needs or if they do somehow it is merely marginal , after a whole day's toil. That is really disheartening , everyday when they get off their homes they really don't know that they would be even able to get enough resources to curb hunger of their family.

  The poor people have become really become hopeless , waiting for a helping hand . Do they deserve this ?
Don't they have any right to contribute to progress of Nation ?  A really sorry sight which I eye-witnessed- Early morning I was at a Railway Station waiting for train to arrive , I bought a cup of tea from a nearby stall and what I noticed was that there was a poor man searching the dustbin attached to the stall curiously but at an interval of  5-10 mins the stall owner scolded him off like a cattle , but the poor guy repeatedly came back .After two or three such encounters he finally set at some distance . Then what I saw was distressful sight that he was fishing for the discarded teabags in the dustbin and he was squeezing that into a cup to get him a cup of morning tea . Seeing that I could not take another sip from my cup , I gave mine to him as well as a packet of biscuit which brought a little smile on his face , which brought down the my guilt marginally.

  I am not suggesting that you start making donations to them or tipping the beggars , no not at  all that will introduce a new shortcut for the poor people for begging.What I learned was that , giving a cup of tea and a packet of biscuit to him , brought him a smile . See the point which I want to make is that they should have their share of  subsidies and aids , which could get some content and some sense of being cared.      

   We hear of things like prices of the general commodities increasing day by day , but the question is that - Are the wages of the poor people rising too? Answer is simply no , the challenge of earning their basic urges is going up by leaps and bounds and they are helpless.Who is going to help them , Me,You or the Government, answer is tough isn't?
 
    We hear of the Government policies and rural development programs , wherein Government sanction huge sum of money for supporting one or other problems like houses for poor, food at subsidized rates ,etc. But one thing to catch is , Are the people at grass root level really getting the money invested upon them , forget about getting do they even know that Government is taking steps to curb their helplessness.
 
   Are these really benefitting the targeted people ?  No , the money or the resources are no doubt flowing from the government treasury , but it has to undergo lots of levels of executives and systems .The layers should be completely porous to let the resources seep, ironically what poor receives are the drops of resources and with such limited resources only limited things can be done , which are still insufficient.But the real concern is that , instead of porous intermediate layers of administration they turn  act semi-porous .
 Why so ? Don't they want upliftment of poor ?

How come the enormous resources turn meagre ?

That is a really controversial point which has a different conclusion for different people so that's a little food for thought for you, feel free to put the comments and suggestions in the box below.

If the poor people are not taken care off, they are already out of league to enhance the growth of nation as they also constitute the growth as a whole , because development of nation means development from the grass root level .

Are the Smartphones really 'SMART' ?

 Today having a Smartphone is a more of a style statement. The comparisons are made with the size of the screens which they have on their phones .The small and stylish phones have now grown into Phablets and even the tablets which are enormously large are coming with calling facility. People have really gone crazy for the smartphones but along with this another trait which gets cultivated along is the obsession or rather addiction to their devices.  

 People nowadays do two things majorly, one that they perform there daily jobs and the second the pastime which they mostly pass on using they so called 'smartphones'. Well day by day the size of the smartphones are getting increased and so is the craze to own one . Things which are aiding the cause is the huge smartphone market which has a policy of selling as much units as possible, and even the price ranges are for people of all the classes. Now as  these smartphones are available to all groups of people this has fueled the craze , and have led them on to a path of being virtual socialization.

The term 'Virtual Socialization' which I have used here refers to endless social messaging apps , and among the social messaging giants are Facebook, Twitter, Whatsapp , etc.   which provide you with amazing power to connect to people (who are in your "Friend List ") virtually with no physical connection , simply OTA (Over The Air ) at your convenience.

 Smartphone user have an edge to this amazing technique , as they can always be logged into their social app accounts and can access anywhere ,anytime it is just the internet pack which they need (which is readily available these days cultivating the cause) and you are good to go.

But the truth is  ,   Is it worth it ?
To overlook the real  available social circle which is powered by real communication than digging deep into your beloved smartphone in order to keep the virtual web around you intact. Moreover  people have grown an habit of searching things online pointlessly just to prove the superiority on others or for just mere fun.

There are even radiation threats on prolonged usage of the smartphones, as in order to make phones smart lots of sensors and receivers are embedded in the smartphones , as we can see that there is lot of competition in the market amongst the different brands the companies tend to use the cheap components which are a emitter of radiations. These radiations could be really very harmful , completely hampering health of the carrier.

People have to think that there is lot more than a phone, rather they should look around and find  things which could be of better good, there is more they can do than keying in the messages and Face-booking as it is said.People have nowadays hobby of  browsing apps in their smartphones and killing their precious time but they need to think harder that virtual socialization is just making virtual connections , which will enable you to believe that you have a vast connection of friends. There might be a case that a person might have over 100 Facebook friends, 50 twitter followers , but in reality he might be struggling to get a one good friend. This is possibly because through social networking sites you might have an edge of making as many friends through sending a friend request but when it comes down to talk in person it could be real hard, but these things are to be understood by each person . Let us leave that topic as its wayward from the title .

People have really stopped observing their surroundings and events happening in their vicinity , because they are busy focussing in their smartphones. People have stopped observing the weather and started asking to 'Siri' and 'Google Now' -- '' How's the weather Like?". People have stopped exploring their surroundings and started finding in the maps  in their phones. People have started texting the smilies without literally exclaiming their emotions , and many more . There are more funny comparisons but the list may just go on. People try to find each and everything in their phone ,  as they spend in them heavily so of course will expect returns and phones in turn don't discourage their hopes providing all kind of virtual solutions and expectations to the user.

I might sound like I totally condemn the usage of smartphone but fellas I do have a smartphone but the thing which is clear in my mind is that it is just another small gadget in my daily life which I use only when it is required ,not using it for a time pass or just a hobby and wasting my precious time in fondling the apps in my phone.


Smartphone ,Yes has an edge over other phones , when we utilize it for daily needs its good but when requirement turns into addiction , then is the time when it hampers the smartness of an individual .

M101 MongoDB for Java Developers' final exam Answer(M101J)

Here I am providing the answers for FINAL exam M101J,which I found out upon solving.Hope you use it wisely, my point of discussing each of the questions is that everyone could check which point are they going wrong and yes I could also get a better solution than mine. So please use it as a extra check after you have solved the question once on your part so that the explanations benefit you the most.


Question 1 : 
Here we need to query the enron dataset calculate the number of messages sent by Andrew Fastow, CFO, to Jeff Skilling, the president. Andrew Fastow's email addess was andrew.fastow@enron.com. Jeff Skilling's email was jeff.skilling@enron.com

So for this first we need to download the enron zip/tar and then import in the mongoDB database name enron and collection name messages . Command for import

mongoimport -d enron -c messages > enron.json

Now switch to mongo Shell  commands:
use enron
db.messages.find({"headers.To":"andrew.fastow@enron.com","headers.From":"jeff.skilling@enron.com"}).count()
This will produce the answer as 3

Question 2:
Please use the Enron dataset you imported for the previous problem. For this question you will use the aggregation framework to figure out pairs of people that tend to communicate a lot. To do this, you will need to unwind the To list for each message.

The mongo shell command which will retrieve the desired answer would be 
db.messages.aggregate([
{
$project: {
from: "$headers.From",
to: "$headers.To"
}
},
{
$unwind: "$to"
},
{
$group : { _id : { _id: "$_id", from: "$from", to: "$to" }
}
},
{
$group : { _id : { from: "$_id.from", to: "$_id.to" }, count: {$sum :1}
}
},
{
$sort : {count:-1}
},
{
$limit: 2
}
])
This would give you the top 2 communication , and check the top most  which would turn out to be :

 "result" : [
         {
                 "_id" : {
                         "from" : "susan.mara@enron.com",
                         "to" : "jeff.dasovich@enron.com"
                 },
                 "count" : 750
         },
         {
                 "_id" : {
                         "from" : "soblander@carrfut.com",
                         "to" : "soblander@carrfut.com"
                 },
                 "count" : 679
         }
 ],

 "ok" : 1
So, it clearly shows the answer is "susan.mara@enron.com" to "jeff.dasovich@enron.com"





Question 3:
In this problem you will update a document in the Enron dataset to illustrate your mastery of updating documents from the shell. Please add the email address "mrpotatohead@10gen.com" to the list of addresses in the "headers.To" array for the document with "headers.Message-ID" of "<8147308.1075851042335.JavaMail.evans@thyme>" 

For this there would be a simple update expression using mongo shell as :
db.messages.update({"headers.Message-ID":"<8147308.1075851042335.JavaMail.evans@thyme>"},{$addToSet:{"headers.To":"mrpotatohead@10gen.com"}})

Then run the validation code and get the validation code as : 897h6723ghf25gd87gh28





Question 4:
Enhancing the Blog to support viewers liking certain comments.

Here you need to work on the BlogPostDAO.java at the area marked as XXXXXX

postsCollection.update(new BasicDBObject("permalink", permalink), new BasicDBObject("$inc", new BasicDBObject("comments." + ordinal + ".num_likes", 1)));

Here in the above command we search the posts collection with the permanent link and increment the like counter by one for the comment which is clicked for like or in other words the ordinal or the order of the comment in the comments array, this ensures that the like is incremented for the comment clicked for like.
Doing this you could see that the like button starts working.

Now run the validator , and you will get the code as : 983nf93ncafjn20fn10f






Question 5 :    

In this question a set of indexes are given and we have to select the indexes which might have been used , in execution of 

db.fubar.find({'a':{'$lt':10000}, 'b':{'$gt': 5000}}, {'a':1, 'c':1}).sort({'c':-1})

 As the Find portion searches on a,b and a,c  and sorting is carries on c reverse order.

_id_ -- This index is not used either in sort or find clause of the operation
a_1_b_1 --  This index is used in the find operation as find is on a,b
a_1_c_1 --  This index is used in the find operation as find is on a,c
c_1 -- This index is also used, because there is a provision that a index is not utilized for the find operation but  for the sort it is used as sort({'c':-1})
a_1_b_1_c_1 - This involves all the three a,b,c and this is also used as it can also be used as a valid index





Question 6
Suppose you have a collection of students of the following form:

{
 "_id" : ObjectId("50c598f582094fb5f92efb96"),
 "first_name" : "John",
 "last_name" : "Doe",
 "date_of_admission" : ISODate("2010-02-21T05:00:00Z"),
 "residence_hall" : "Fairweather",
 "has_car" : true,
 "student_id" : "2348023902",
 "current_classes" : [
  "His343",
  "Math234",
  "Phy123",
  "Art232"
 ]
}

Now suppose that basic inserts into the collection, which only include the last name, first name and student_id, are too slow. What could potentially improve the speed of inserts. Check all that apply.

Add an index on last_name, first_name if one does not already exist.
Set w=0, j=0 on writes
Remove all indexes from the collection
Provide a hint to MongoDB that it should not use an index for the inserts
Build a replica set and insert data into the secondary nodes to free up the primary nodes.

option 1 - As a fact adding index affects reading not writing so it would be indifferent with the indexing so not this option
Option 2 seems to be valid as when w=0 and j=0 is done for the writes no waiting is done at all are no wait is required to obtain as the write confirmations , simply the data is dumped without verification therefore speeding the writes
Option 3 removing indexes would actually help as it would reduce the load and speed up the writing process
Option 4 This seems absurd
Option 5 This is not actually possible as writes are not possible on the secondary nodes so not valid option






Question 7
You have been tasked to cleanup a photosharing database. The database consists of two collections, albums, and images. Every image is supposed to be in an album, but there are orphan images that appear in no album. Here are some example documents (not from the collections you will be downloading). 

When you are done removing the orphan images from the collection, there should be 90,017 documents in the images collection. 

In order to remove the Orphans talked I wrote a Java Program :

/**
 *
 * @author Ankur Gupta
 */
public class Test {
    
        public static void main(String[] args) throws IOException {
            MongoClient c =  new MongoClient(new MongoClientURI("mongodb://localhost"));
            DB db = c.getDB("finaltask");
            int i =0;
            DBCollection album = db.getCollection("albums");
            DBCollection image = db.getCollection("images");
            
            DBCursor cur = image.find();
            cur.next();
            
            while (cur.hasNext()){
                Object id = cur.curr().get("_id");
               DBCursor curalbum = album.find(new BasicDBObject("images", id));
               if(!curalbum.hasNext()){
                   image.remove(new BasicDBObject("_id", id));
               }
               cur.next();
            }
        }
}

In order to verify above statement after removing orphans :
db.albums.aggregate({$unwind:"$images"},{$group:{_id:null,sum:{$sum:"$images"},count:{$sum:1}}})
The result looks like:
    "result" : [
            {
                    "_id" : null,
                    "sum" : NumberLong("4501039268"),
                    "count" : 90017
            }
    ],
    "ok" : 1

To prove you did it correctly, what are the total number of images with the tag 'sunrises" after the removal of orphans?
db.images.find({"tags":"sunrises"}).count()
This will fetch the final answer as  45044




Question 8:
Supposed we executed the following Java code. How many animals will be inserted into the "animals" collection?

public class Question8 {



        public static void main(String[] args) throws IOException {
            MongoClient c =  new MongoClient(new MongoClientURI("mongodb://localhost"));
            DB db = c.getDB("test");
            DBCollection animals = db.getCollection("animals");


            BasicDBObject animal = new BasicDBObject("animal", "monkey");

            animals.insert(animal);
            animal.removeField("animal");
            animal.append("animal", "cat");
            animals.insert(animal);
            animal.removeField("animal");
            animal.append("animal", "lion");
            animals.insert(animal);

        }

}

When you run the above , then you will see an error is thrown that there is a duplicate ID , as we are trying to add , documents again and again on the same Id as we are modifying the same document . So the only one document will be inserted in the collection which will be the first insert as {_id::xxx,"animal","monkey"}
then when again ("animal","cat") is tried to push then the id is same so , it throws duplicate key . So answer is that only one document gets inserted.




Question 9:
Imagine an electronic medical record database designed to hold the medical records of every individual in the United States. Because each person has more than 16MB of medical history and records, it's not feasible to have a single document for every patient. Instead, there is a patientcollection that contains basic information on each person and maps the person to a patient_id, and arecord collection that contains one document for each test or procedure. One patient may have dozens or even hundreds of documents in the record collection. 

We need to decide on a shard key to shard the record collection. What's the best shard key for therecord collection, provided that we are willing to run scatter gather operations to do research and run studies on various diseases and cohorts? That is, think mostly about the operational aspects of such a system.

patient_id
_id
primary care physican (your principal doctor)
date and time when medical record was created
patient first name
patient last name

Here among the options given for the shard key most favourable is patient_id , as there are large number of patient_id and they have been distributed in different diseases, and when a scatter gather operation is carried out then the data is far more expanded on the basis of patient_id.

Other options are not suitable for the scatter and gather operation.




Question 10:
Understanding the output of explain We perform the following query on the enron dataset:

db.messages.find({'headers.Date':{'$gt': new Date(2001,3,1)}},{'headers.From':1, _id:0}).sort({'headers.From':1}).explain()

and get the following explain output.

{
 "cursor" : "BtreeCursor headers.From_1",
 "isMultiKey" : false,
 "n" : 83057,
 "nscannedObjects" : 120477,
 "nscanned" : 120477,
 "nscannedObjectsAllPlans" : 120581,
 "nscannedAllPlans" : 120581,
 "scanAndOrder" : false,
 "indexOnly" : false,
 "nYields" : 0,
 "nChunkSkips" : 0,
 "millis" : 250,
 "indexBounds" : {
  "headers.From" : [
   [
    {
     "$minElement" : 1
    },
    {
     "$maxElement" : 1
    }
   ]
  ]
 },
 "server" : "Andrews-iMac.local:27017"
}

The query did not utilize an index to figure out which documents match the find criteria.

The query used an index for the sorting phase.

The query returned 120,477 documents

The query performed a full collection scan

Here the correct options will be :
Option 1 seems to be correct as  if you could notice that "cursor" : "BtreeCursor headers.From_1" that means that headers.From_1 is used which is not in the find clause but is in the sorting

Option 2 also seems to be correct as "cursor" : "BtreeCursor headers.From_1" the cursor is used in the sorting phase

Option 3 This option is wrong as 83057 records as n=83057

Option 4 This option is correct as if we see nscanned objects is 120477 so it has scanned all



Hope that above explanation prove helpful, please put your precious comments and suggestions on better method to do any question.

Pentaho - A Powerful Business Intelligence Tool

The Pentaho BI Project is open source application software for enterprise reporting, analysis, dashboard, data mining, workflow and ETL. There are many business analytics vendors available but, what’s different in Pentaho is that it brings together IT and business users to access, integrate, blend, visualize and analyze all data that impacts business results. As pentaho has a commercial open source license, open source heritage drives continued innovation in a modern, unified, embeddable analytics platform that is purpose-built to save time and money. Moreover Pentaho is an integrated platform. Most business analytics vendors do some range of reporting and analytics; and actually the data integration process is combined with the analytical toolset, saving customers time, money and getting faster time to value. Pentaho is developed entirely on java and has and Open Web-Based API’s and has a unique pluggable Architecture

Pentaho is open source based BI suite offering Reporting, ETL, Dashboards and Data Mining capabilities. It offers free community edition with online community support through forums and wikis and an enterprise suite encompassing all BI modules. The Pentaho platform can be used for business Analytics, data mining , data integration, big data and for analysis services .Now further will talk about each of the utility and the platform provided by Pentaho to implement the utility briefly

Pentaho Big Data Analytics –

Within a single platform it provides visual tools to extract and prepare our data plus the visualizations and analytics that will change the way we run our business.  Regardless of the data source, analytic requirement or deployment environment, Pentaho allows to turn big data into big insights. A tightly coupled data integration and business analytics platform accelerates the realization of value from big data.

                  

Pentaho Business Analytics-

Pentaho's modern, simplified and interactive approach empowers business users to access, discover and blend all types and sizes of data.  With a spectrum of increasingly advanced analytics, from basic reports to predictive modeling, users can analyze and visualize data across multiple dimensions, all while minimizing dependence on IT.  At the same time, a true designed-for-mobile experience ensures users are productive no matter where they are.The Pentaho Business Analytics suite provides a full spectrum of data integration and business intelligence (BI) capabilities including ETL, OLAP, query and reporting, interactive analysis, dashboards, data mining and a BI platform that has made it the world's most popular open source BI suite. Pentaho's platform also provides broad enterprise data services including integration with Hadoop for big data analytics and support for the company's Agile BI initiative, which enables organizations to build BI applications more quickly, respond to business changes more easily and expedite time-to-value for up to 90% less cost than traditional BI vendors.

Pentaho Data Integration-

With Pentaho Data Integration, Pentaho is redefining the way that BI applications are built and deployed. Utilizing Pentaho’s Agile BI approach, Pentaho Data Integration unifies the ETL, modeling and visualization processes into a single, integrated environment that enables developers and end-users to work seamlessly together.  The end result is that BI developers and end users can build BI applications more quickly, easily and at a small fraction of the cost of traditional solutions.

Pentaho Analysis Services(Modrian)-

Pentaho Analysis Services Community Edition also known as Mondrian. Mondrian is an Online Analytical Processing (OLAP) server that enables business users to analyze large quantities of data in real-time. Users explore business data by drilling into and cross-tabulating information with speed-of-thought response times to complex analytical queries. Pentaho Analyzer provides intuitive, interactive analytical reporting letting non-technical business users quickly understand business information. As part of the enhanced functionality in Pentaho Analysis Enterprise Edition, Analyzer features:

·         Web-based, drag-and-drop report creation

·         Advanced sorting and filtering

·         Drill through reports into the underlying data

·         Chart visualizations including conditional stop-lighting

Pentaho Data Mining (Weka)-

Pentaho Data Mining Community Edition (CE) also known as Weka. Pentaho Data Mining is a comprehensive set of tools for machine learning and data mining. Its broad suite of classification, regression, association rules and clustering algorithms can be used to help you understand the business better and also be exploited to improve future performance through predictive analytics.

The main focus of this article is mainly on Pentaho Report Designer (PRD), Pentaho Data Integration (PDI) and Pentaho Bi-Server. These are readily available in the community edition pack of Pentaho BI Suite Community Edition (CE).Further with throw light on each of them individually.

Pentaho Report Designer(PRD) -

Pentaho Reporting is a suite of tools for creating pixel perfect reports. With Pentaho Reporting we are able to transform data into meaningful information tailored to concerned audience. We can create HTML, Excel, and PDF, Text or printed reports. If you are a developer, you can also produce CSV and XML reports to feed other systems. It provides a very simple interface to create report and very easy to use features. Pentaho reporting is a powerful tool to create reports, it provides great connectivity with vivid options to connect to various data sources like MySQL, Oracle, Mongo DB, Java Beans and lot more. It provides various designing options which helps to portray your ideas to reality with just a snap of finger and without compromising on the reliability. Further it provides wide range of export options like PDF, XLS, XML, and many more. We can even add various kind of charts which provide better portrayal of the data. Reports made in the PRD are dynamic as it could take input from the end user in terms of parameters and create report on the fly with those parameters.  For designing a report following simple steps could be followed – 

1.      Define a data source      

      2.      Determine the query to retrieve fields from the data source

      3.      Design the Report with the tools provided.

      4.      Export the report in required format.

Pentaho BI-Server:

The Pentaho BI Platform provides the architecture and infrastructure required to build solutions to business intelligence (BI) problems. The framework provides core services including authentication, logging, auditing, web services, and rules engines. The platform also includes a solution engine that integrates reporting, analysis, dashboards and data mining components. The modular design and plugin based architecture allows all or part of the platform to be embedded into third party applications by end users as well as OEMs Through Pentaho Server you can create Ad-Hoc Reports easily , with only few simple steps. Following simple steps could be followed in order to create a report –

1.      Define the data source

2.      Select the template

3.      Select the fields and put them into specified bands using Drag and Drop , even we could add the constraints and Groups

4.      Next we could customize the Selections of the fields like setting the formats and applying filters, functions (like count, sum etc.) ,sorting and alignment

5.      Next we could specify Orientation and the Paper for the Report and could key in the description for the report , as well as the Header and Footer for the Report and Page could be specified here

6.      Now we are ready with the report , at the bottom there is a drop down which says Preview As and options are HTML/Excel/CSV and after choosing one of them we could hit Go to view the report run in the specified format.

Pentaho Data Integration(PDI):

Pentaho Data Integration Community Edition (PDI CE) also known as Kettle. Pentaho Data Integration delivers powerful Extraction, Transformation and Loading (ETL) capabilities using an innovative, metadata-driven approach. With an intuitive, graphical, drag and drop design environment, and a proven, scalable, standards-based architecture, Pentaho Data Integration is increasingly the choice for organizations over traditional, proprietary ETL or data integration tools. Pentaho Data Integration provides support for slowly changing dimensions and surrogate key for data warehousing, allows data migration between databases and application, is flexible enough to load giant datasets, and can take full advantage of cloud, clustered, and massively parallel processing environments. We can cleanse your data using transformation steps that range from very simple to very complex. Pentaho Data Integration provides tools that include ETL, modeling, and visualization in one unified environment — with the help of Spoon interface. Basically there are two main components Transformations and Jobs.

        

There are basically two building blocks which leads the course of action — Transformations and Jobs .

Transformations:

A transformation is a network of logical tasks called steps. Transformations are essentially data flows. For instance , a developer creates a flat CSV file with enormous data, and other database developer has to put a part of data of the CSV file in a database filtering all the other data and as an extra he wants to keep track of the mismatch records in a file, so basically it would be done as :

                                                         

The two main components associated with transformations are steps and hops. Steps are the building blocks of a transformation, for example a text file input or a table output whereas Hops are data pathways that connect steps together and allow schema metadata to pass from one step to another.

Jobs:

Jobs are workflow-like models for coordinating resources, execution, and dependencies of ETL activities. For instance a simple Job would be to load a data from a data file, like shown below Jobs starts with waiting for the data file then it loads the data at desired location and if the data successfully loaded then heads to success or goes to error log.

                                                             

Pentaho in a nutshell is a very powerful platform in order to perform Business Intelligence Operations. Please put suggestions and questions in  the comments section I will be more than happy to answer them.

Bullies of the corporate world!!!!!

Actually  if anyone would think about their past lives - schooling days, college days.....  There  is a prominent appearance of a unwanted person known as BULLIES
  Someone might have been a bully who would have acted as a roadblock. But as we move onto a corporate world then we think those hard times given by the bullies have gone. But,  heads up  its really sad to see the same bullies in the corporate world too.

   Do you think people at higher posts  have become meaner  and are sunk in there pride??

Well it seems that  these people become bully because they have been bullied in the beginning  of their career by their seniors and later when they are in a respectable position  they desire to bring the same wrath upon the new comers.

Because of this unwelcoming nature of the senior workers, newbies have a hard time to blend in the office culture and also could not provide their cent percent in the task allotted to them.  Mostly they have fear if a thing is done wrongly once then all the blame is put upon them without giving a thought to the mental stress , will he be able to bear the blame or just perish under the blame and get exposed to the sarcasm .  People at the high post just to clear their hands of the blame they plant the blame on the immediate junior and then he does to the other  level and so the chain of imposition of blame continues down the hierarchy.

So it's not a person but the whole chain of bullies being followed. But If the people at good position so called bosses,  team leads,  group leader somebody  takes the stand and does not bully the new comers, then he will definitely  give space for new people who think in a progressive  way. This will help  in reducing the bully concept from the system. If the mangers, team leads understand the error and impose the blame on not only on the member at the lowest level but he should impose it in the team completely so that  whole team of people are held responsible.

Yes someone might say that what will happen if one manager or lead does that but news flash, this small stand will help the juniors or the new comers prosper and would enhance the office environment.

So this is my view stating the problems but others might have a different opinion. I might have offended some people sorry for that, but someone needs to come forward  for this.