You have a table called scholar where there are two fields id and create date. You...

You have a table called scholar where there are two fields id and create date. You want to create an ETL process that loads the data from scholar table into the big data platform. What are the different things that you will put on a requirement document for the developer to write code and implement the solution?

Solutions

Expert Solution

Answer:

We dont hve any infromation about big data model in question so below document will have a generalized approach.

Requirement Document

1. Extraction: Since we have only 2 parameters. We will have a DB call according to the range of Ids which will fetch the created date filled for the records.

2. Load: We will have the extracted records in the form of the list and we will feed the list to the big data model(list is easily iterable and we have few fields to handle).

3.Transform: The big data model will give us the response against the fed data (which will be the analysis of the data). which will be in the form of dictionary and will be stored in the new DB table in the form of created date against the anlaysed data (created date{date time field}, model_response{dictionary}).

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

# Feel free to ask questions in the comment section

# Hit the like if my answer if it is worth helping to you.

# you can payback and help me with a like.

venereology answered 1 year ago

Next > < Previous

Related Solutions

Using SQL create a new database called school_app. Create a student table with fields id (auto...

Using SQL create a new database called school_app. Create a student table with fields id (auto increment), first_name, last_name. Create a course table with fields id (auto increment), course code (such as ITC or MTH), and course number (such as 100 or 295). Note that the relationship between student and course is many-to-many (n:m). Create a join table called student_course that implements the n:m relationship with fields id (auto increment), student_id, course_id, and grade (which has values 0, 1, 2,...

Create a table ‘StudentInfo’ with following fields: ID First Name Last Name SSN Date of Birth...

Create a table ‘StudentInfo’ with following fields: ID First Name Last Name SSN Date of Birth Create a table ‘ClassInfo’ table: ID Class Name Class Description Create a table ‘RegisteredClasses’ table: StudentID ClassID The RegisteredClasses table should have a foreign key relationship to StudentInfo and ClassInfo tables for the respective IDs. Also the IDs in StudentInfo and ClassInfo need to be primary keys. When you submit the file your email should also contain the following SQL Queries: Query to show...

Create a class, called Song. Song will have the following fields:  artist (a string) ...

Create a class, called Song. Song will have the following fields:  artist (a string)  title (a string)  duration (an integer, recorded in seconds)  collectionName (a string) All fields, except for collectionName, will be unique for each song. The collectionName will have the same value for all songs. In addition to these four fields, you will also create a set of get/set methods and a constructor. The get/set methods must be present for artist, title, and duration....

Create an Automobile class for a dealership. Include fields for an ID number, make, model, color,...

Create an Automobile class for a dealership. Include fields for an ID number, make, model, color, year, vin number, miles per gallon, and speed. Include get and set methods for each field. Do not allow the ID to be negative or more than 9999; if it is, set the ID to 0. Do not allow the year to be earlier than 2000 or later than 2017; if it is, set the year to 0. Do not allow the miles per...

Create a class named Salesperson. Data fields for Salesperson include an integer ID number and a...

Create a class named Salesperson. Data fields for Salesperson include an integer ID number and a doubleannual sales amount. Methods include a constructor that requires values for both data fields, as well as get and set methods for each of the data fields. Write an application named DemoSalesperson that declares an array of 10 Salesperson objects. Set each ID number to 9999 and each sales value to zero. Display the 10 Salesperson objects. public class DemoSalesperson { public static void...

Create a class named Student. Student has fields for an ID number, number of credit hours...

Create a class named Student. Student has fields for an ID number, number of credit hours earned, and number of points earned. (For example, many schools compute grade point averages based on a scale of 4, so a three-credit-hour class in which a student earns an A is worth 12 points.) Include methods to assign values to all fields. Student also has a field for grade point average. Include a method to compute the grade point average field by dividing...

Consider the following relation AIRLINE TABLE Flight Date Flight ID Origin Destination Captain ID Captain Name...

Consider the following relation AIRLINE TABLE Flight Date Flight ID Origin Destination Captain ID Captain Name Number of Passengers on the Flight April 10 111 Chicago L.A. 111 Tracy 110 April 10 222 L.A. Phoenix 222 Matt 105 April 12 111 Chicago L.A. 444 Tim 98 April 12 333 L.A. N.Y. 111 Tracy 75 April 12 444 L.A. N.Y. 333 Pat 110 April 12 555 L.A. Dallas 444 Tim 111 April 12 222 L.A. Phoenix 555 Pat 107 Some of...

You have a table for a membership database that contains the following fields: MemberLastName,

You have a table for a membership database that contains the following fields: MemberLastName, MemberFirstName, Street, City, State, ZipCode, and InitiationFee. There are 75,000 records in the table. What indexes would you create for the table, and why would you create these indexes?

Define a table with the following properties that represent an election. Create a unique id for...

Define a table with the following properties that represent an election. Create a unique id for the table. Create a field or fields representing an election's description. Create a field or fields representing an election's date. Create a field or fields representing an election's status (active or inactive)

Create a very simple temperature converter form having two text fields. The first one is where...

Create a very simple temperature converter form having two text fields. The first one is where the user will enter temperature. The second field is where the computed temperature value is displayed depending on the unit (F or C).use html and javascript

Subjects