I have implement two classes, named a Ticket and Flight.
I have the main program is FlightTicketsProgram (at downside):
ArrayList<Ticket> tickets = new ArrayList<Ticket>();
ArrayList<Flight> flights = new ArrayList<Flight>();
Ticket ticket1 = new Ticket("201409112003", "George", "White", "TK2141");
Ticket ticket2 = new Ticket("201408182023", "John", "Clemens", "TK1852");
Ticket ticket3 = new Ticket("201409171004", "Alex", "Cats", "TK2141");
tickets.add(ticket1);
tickets.add(ticket2);
tickets.add(ticket3);
Flight flight1 = new Flight("TK2141", "THY", "Ankara", "Istanbul", "25.08.2014", "14:00", "14:50");
Flight flight2 = new Flight("TK1852", "THY", "Barselona", "Istanbul", "15.09.2014", "05:55", "10:20");
Flight flight3 = new Flight("PC2136", "Pegasus", "Adana", "Izmir", "18.07.2014", "20:35", "22:40");
Flight flight4 = new Flight("DL9450", "Delta", "Frankfurt", "Amsterdam", "20.09.2014", "10:55", "13:00");
Flight flight5 = new Flight("KL1857", "KLM", "Amsterdam", "Dusseldorf", "13.08.2014", "13:20", "14:30");
Flight flight6 = new Flight("PC1177", "Pegasus", "Antalya", "Istanbul", "22.10.2014", "10:00", "12:40");
flights.add(flight1);
flights.add(flight2);
flights.add(flight3);
flights.add(flight4);
flights.add(flight5);
flights.add(flight6);
//Step 1: print tickets
System.out.println("Tickets:");
System.out.print(ticket1);
System.out.print(ticket2);
System.out.print(ticket3);
//Step 2: print flights
System.out.println("\nExisting flights:");
System.out.println(flight1);
System.out.println(flight2);
System.out.println(flight3);
System.out.println(flight4);
System.out.println(flight5);
//Step 3: change a ticket
System.out.println("Ticket change:");
tickets.get(0).setFlightNumber("PC2136");
System.out.print(tickets.get(0));
//Step 4: change a flight
System.out.println("\nFlight change:");
flights.get(5).setDeparture("10:20");
flights.get(5).setArrival("13:00");
System.out.print(flights.get(5));
I have to see my output like at downside for FlightTicketsProgram.java.
Tickets:
201409112003 - TK2141 - George White
201408182023 - TK1852 - John Clemens
201409171004 - TK2141 - Alex Cats
Existing flights:
THY - TK2141 Date: 25.08.2014
From: Ankara - To: Istanbul
Departure: 14:00 - Arrival: 14:50
How can I do this?
use toString() in your classes Flight and Ticket
Related
Is there a way to join 2 dataset without explode rows? I need only a flag if at least one row of dataset "df2" satisfies the join condition with the dataset of "df1".
Is there any way to avoid the join? I would like to avoid joining and then just keep the first row with a window function.
Condition left join is = [(df2.id == df1.id) & (df2.date >= df1.date)]
Example:
Input df1
id
city
sport_event
date
abc
London
football
2022-02-11
def
Paris
volley
2022-02-10
ghi
Manchester
basketball
2022-02-09
Input df2
id
num_spect
date
abc
100.000
2022-01-10
abc
200.000
2022-04-15
abc
150.000
2022-02-11
Output NOT DESIDERED <- NOT DESIDERED
id
city
sport_event
date
num_spect
abc
London
football
2022-02-11
100.000
abc
London
football
2022-02-11
200.000
abc
London
football
2022-02-11
150.000
def
Paris
volley
2022-02-10
ghi
Manchester
basketball
2022-02-09
Output DESIDERED <- DESIDERED
id
city
sport_event
date
num_spect
flag
abc
London
football
2022-02-11
100.000
1
def
Paris
volley
2022-02-10
ghi
Manchester
basketball
2022-02-09
Here's my implementation using left join
from pyspark.sql import functions as F
from pyspark.sql.types import *
from pyspark.sql import Window
df1 = spark.createDataFrame(
[
("abc", "London", "football", "2022-02-11"),
("def", "Paris", "volley", "2022-02-10"),
("ghi", "Manchester", "basketball", "2022-02-09"),
],
["id", "city", "sport_event", "date"],
)
df1 = df1.withColumn("date", F.col("date").cast(DateType()))
df2 = spark.createDataFrame(
[
("abc", "100.000", "2022-01-10"),
("abc", "200.000", "2022-04-15"),
("abc", "150.000", "2022-02-11"),
],
["id", "num_spect", "date"],
)
df2 = (df2
.withColumn("num_spect", F.col("num_spect").cast(DecimalType(18,3)))
.withColumn("date", F.col("date").cast(DateType()))
)
row_window = Window.partitionBy(
"df1.id",
"city",
"sport_event",
"df1.date",
).orderBy(F.col("num_spect").asc())
final_df = (
df1.alias("df1")
.join(
df2.alias("df2"),
on=(
(F.col("df1.id") == F.col("df2.id"))
& (F.col("df2.date") >= F.col("df1.date"))
),
how="left",
)
.withColumn(
"flag",
F.when(
F.col("df2.id").isNull(),
F.lit(None),
).otherwise(F.lit(1)),
)
.withColumn("row_num", F.row_number().over(row_window))
.filter(F.col("row_num") == 1)
.orderBy(F.col("df1.id"))
.drop(F.col("df2.id"))
.drop(F.col("df2.date"))
.drop(F.col("row_num"))
)
final_df.show()
OUTPUT:
+---+----------+-----------+----------+---------+----+
| id| city|sport_event| date|num_spect|flag|
+---+----------+-----------+----------+---------+----+
|abc| London| football|2022-02-11| 150.000| 1|
|def| Paris| volley|2022-02-10| null|null|
|ghi|Manchester| basketball|2022-02-09| null|null|
+---+----------+-----------+----------+---------+----+
Here are my 2 cents:
1.Create the data frames as follows:
df1 = spark.createDataFrame(
[("abc", "London", "football", "2022-02-11"),("def", "Paris", "volley", "2022-02-10"),("ghi", "Manchester", "basketball", "2022-02-09")],
schema = ["id", "city", "sport_event", "date"]
).withColumn("date", fx.col("date").cast(DateType()))
df2 = spark.createDataFrame(
[("abc", "100.000", "2022-01-10"),("abc", "200.000", "2022-04-15"),("abc", "150.000", "2022-02-11")],
schema = ["id", "num_spect", "date"]
) .withColumn("num_spect", fx.col("num_spect").cast(DecimalType(18,3)))\
.withColumn("date", fx.col("date").cast(DateType())) .orderBy(col('date'))
Then join based on the condition and then drop duplicates, finally update the column flag based on num_spect value:
df_joined = df1.join(df2, [(df1.id == df2.id) & (df1.date >= df2.date)], how = 'left').drop(df2.date).drop(df2.id).dropDuplicates(["id", "city", "sport_event", "date"])
df_joined = df_joined.withColumn('flag', when(col('num_spect').isNotNull(),1).otherwise(0))
Print the data frame
df_joined.show()
Please refer the below screenshot for your reference:
page1 and page2 URL. I want to fetch all the content from the 1st URL and only the main text from the 2nd URL and append it to the main text of 1st URL. This is only one article. function parse_indianexpress_archive_links() contains a list of news articles URLs. I m getting all the results from page1 but the next_maintext column from page2 results output <GET http://archive.indianexpress.com/news/congress-approves-2010-budget-plan/442712/2>
class spider_indianexpress(scrapy.Spider):
name = 'indianexpress'
start_urls = parse_indianexpress_archive_links()
def parse(self,response):
items = ScrapycrawlerItem()
separator = ''
#article_url = response.xpath("//link[#rel = 'canonical']/#href").extract_first()
article_url = response.request.url
date_updated = max(response.xpath("//div[#class = 'story-date']/text()").extract() , key=len)[-27:] #Call max(list, key=len) to return the longest string in list by comparing the lengths of all strings in a list
if len(date_updated) <=10:
date_updated = max(response.xpath("//div[#class = 'story-date']/p/text()").extract() , key=len)[-27:]
headline = response.xpath("(//div[#id = 'ie2013-content']/h1//text())").extract()
headline=separator.join(headline)
image_url = response.css("div.storybigpic.ssss img").xpath("#src").extract_first()
maintext = response.xpath("//div[#class = 'ie2013-contentstory']//p//text()").extract()
maintext = ' '.join(map(str, maintext))
maintext = maintext.replace('\r','')
contd = response.xpath("//div[#class = 'ie2013-contentstory']/p[#align = 'right']/text()").extract_first()
items['date_updated'] = date_updated
items['headline'] = headline
items['maintext'] = maintext
items['image_url'] = image_url
items['article_url'] = article_url
next_page_url = response.xpath("//a[#rel='canonical']/#href").extract_first()
if next_page_url :
items['next_maintext'] = scrapy.Request(next_page_url , callback = self.parse_page2)
yield items
def parse_page2(self, response):
next_maintext = response.xpath("//div[#class = 'ie2013-contentstory']//p//text()").extract()
next_maintext = ' '.join(map(str, next_maintext))
next_maintext = next_maintext.replace('\r','')
yield {next_maintext}
Output:
article_url,date_publish,date_updated,description,headline,image_url,maintext,next_maintext
http://archive.indianexpress.com/news/congress-approves-2010-budget-plan/442712/,,"Fri Apr 03 2009, 14:49 hrs ",,Congress approves 2010 budget plan,http://static.indianexpress.com/m-images/M_Id_69893_Obama.jpg,"The Democratic-controlled US Congress on Thursday approved budget blueprints embracing President Barack Obama's agenda but leaving many hard choices until later and a government deeply in the red. With no Republican support, the House of Representatives and Senate approved slightly different, less expensive versions of Obama's $3.55 trillion budget plan for fiscal 2010, which begins on October 1. The differences will be worked out over the next few weeks. Obama, who took office in January after eight years of the Republican Bush presidency, has said the Democrats' budget is critical to turning around the recession-hit US economy and paving the way for sweeping healthcare, climate change and education reforms he hopes to push through Congress this year. Obama, traveling in Europe, issued a statement praising the votes as ""an important step toward rebuilding our struggling economy."" Vice President Joe Biden, who serves as president of the Senate, presided over that chamber's vote. Democrats in both chambers voted down Republican alternatives that focused on slashing massive deficits with large cuts to domestic social spending but also offered hefty tax breaks for corporations and individuals. ""Democrats know that those policies are the wrong way to go,"" House Majority Leader Steny Hoyer told reporters. ""Our budget lays the groundwork for a sustained, shared and job-creating recovery."" But Republicans have argued the Democrats' budget would be a dangerous expansion of the federal government and could lead to unnecessary taxes that would only worsen the country's long-term fiscal situation. ""The Democrat plan to increase spending, to increase taxes, and increase the debt makes no difficult choices,"" said House Minority Leader John Boehner. ""It's a roadmap to disaster."" The budget measure is nonbinding but it sets guidelines for spending and tax bills Congress will consider later this year. BIPARTISANSHIP ABSENT AGAIN Obama has said he hoped to restore bipartisanship when he arrived in Washington but it was visibly absent on Thursday. ... contd.",<GET http://archive.indianexpress.com/news/congress-approves-2010-budget-plan/442712/2>
This is not how Scrapy works (I mean next_page request) How to fetch the Response object of a Request synchronously on Scrapy?.
But in fact you don't need synchronous requests. All you need is to check for a next page and pass current state (item) to the callback that will process your next page. I'm using cb_kwargs (it's a recommended way now). You may need to use request.meta if you have an old version.
import scrapy
class spider_indianexpress(scrapy.Spider):
name = 'indianexpress'
start_urls = ['http://archive.indianexpress.com/news/congress-approves-2010-budget-plan/442712/']
def parse(self,response):
item = {}
separator = ''
#article_url = response.xpath("//link[#rel = 'canonical']/#href").extract_first()
article_url = response.request.url
date_updated = max(response.xpath("//div[#class = 'story-date']/text()").extract() , key=len)[-27:] #Call max(list, key=len) to return the longest string in list by comparing the lengths of all strings in a list
if len(date_updated) <=10:
date_updated = max(response.xpath("//div[#class = 'story-date']/p/text()").extract() , key=len)[-27:]
headline = response.xpath("(//div[#id = 'ie2013-content']/h1//text())").extract()
headline=separator.join(headline)
image_url = response.css("div.storybigpic.ssss img").xpath("#src").extract_first()
maintext = response.xpath("//div[#class = 'ie2013-contentstory']//p//text()").extract()
maintext = ' '.join(map(str, maintext))
maintext = maintext.replace('\r','')
contd = response.xpath("//div[#class = 'ie2013-contentstory']/p[#align = 'right']/text()").extract_first()
item['date_updated'] = date_updated
item['headline'] = headline
item['maintext'] = maintext
item['image_url'] = image_url
item['article_url'] = article_url
next_page_url = response.xpath('//a[#rel="canonical"][#id="active"]/following-sibling::a[1]/#href').extract_first()
if next_page_url :
yield scrapy.Request(
url=next_page_url,
callback = self.parse_next_page,
cb_kwargs={
'item': item,
}
)
else:
yield item
def parse_next_page(self, response, item):
next_maintext = response.xpath("//div[#class = 'ie2013-contentstory']//p//text()").extract()
next_maintext = ' '.join(map(str, next_maintext))
next_maintext = next_maintext.replace('\r','')
item["maintext"] += next_maintext
next_page_url = response.xpath('//a[#rel="canonical"][#id="active"]/following-sibling::a[1]/#href').extract_first()
if next_page_url :
yield scrapy.Request(
url=next_page_url,
callback = self.parse_next_page,
cb_kwargs={
'item': item,
}
)
else:
yield item
I have a datatable contains three columns (Names and Dates)
Names Dates amount
----------------------------------
John 01/01/2019 5
John 02/01/2019 10
John 04/01/2019 5
John 05/01/2019 4
Adam 01/01/2019 5
Adam 03/01/2019 5
Adam 04/01/2019 5
I need to check missed days and make periods
I expect output like
Names Fr To amount
John 01/01/2019 02/01/2019 15
John 04/01/2019 05/01/2019 9
Adam 01/01/2019 01/01/2019 5
Adam 03/01/2019 04/01/2019 10
If you divide the problem into sub-parts then it becomes a lot simpler.
Make pairs from the entries of the datatable.
Now make groups based on Name.
On this groups check which group has the time difference of 1 month.
Project this groups using a foreach loop.
Now the code,
public static class Program
{
static int CountNumberOfMonths(DateTime date1, DateTime date2) => (date2.Month - date1.Month) + 12 * (date2.Year - date1.Year);
static void Main(string[] args)
{
DataTable table = new DataTable();
table.Columns.Add("Name", typeof(string));
table.Columns.Add("Dates", typeof(DateTime));
table.Columns.Add("Amount", typeof(int));
table.Rows.Add("John", new DateTime(2019, 1, 1), 5);
table.Rows.Add("John", new DateTime(2019, 2, 1), 10);
table.Rows.Add("John", new DateTime(2019, 4, 1), 5);
table.Rows.Add("John", new DateTime(2019, 5, 1), 4);
table.Rows.Add("Adam", new DateTime(2019, 1, 1), 5);
table.Rows.Add("Adam", new DateTime(2019, 3, 1), 5);
table.Rows.Add("Adam", new DateTime(2019, 4, 1), 5);
var enumeratedDatatable = table.AsEnumerable();
var filteredData = enumeratedDatatable
.Zip(enumeratedDatatable.Skip(1), Tuple.Create)
.GroupBy(x => x.Item1.Field<string>("Name"))
.Select(x => x.Where(y =>
{
int numMonths = CountNumberOfMonths(y.Item1.Field<DateTime>("Dates"), y.Item2.Field<DateTime>("Dates"));
return numMonths > 0 && numMonths == 1;
}));
DataTable result = new DataTable();
result.Columns.Add("Name", typeof(string));
result.Columns.Add("From", typeof(DateTime));
result.Columns.Add("Sum", typeof(DateTime));
result.Columns.Add("Amount", typeof(int));
foreach (var item in filteredData)
{
string name = item.ElementAt(0).Item1.Field<string>("Name");
foreach (var innerItem in item)
{
int sumAmount = innerItem.Item1.Field<int>("Amount") + innerItem.Item2.Field<int>("Amount");
DateTime from = innerItem.Item1.Field<DateTime>("Dates");
DateTime to = innerItem.Item2.Field<DateTime>("Dates");
result.Rows.Add(name, from, to, sumAmount);
}
}
// Printing the results to standard output
foreach (DataRow row in result.Rows)
{
string rowItem = string.Join("\t", row.ItemArray);
Console.WriteLine(rowItem);
}
}
}
Outputting:
John 01/01/2019 00:00:00 02/01/2019 00:00:00 15
John 04/01/2019 00:00:00 05/01/2019 00:00:00 9
Adam 03/01/2019 00:00:00 04/01/2019 00:00:00 10
I'm fetching rows from excel sheet in my application that holds attendance records from the bio metric machine. In order to get the best result i have to remove the redundant data. For that I have to manage check in and checkout timings at regular intervals. For instance, First check in time for entering, and then checkout time for lunch, then again check in for returning back, and last check out for going home. Meanwhile the rows in excel contains multiple check ins and check outs as the employee tends to do more that once for both.
I have managed to get records from excel and added to data table. Now for the sequence and sorting part I'm struggling to achieve my desired result. Below is my code.
protected void btnSaveAttendance_Click(object sender, EventArgs e)
{
try
{
if (FileUpload1.HasFile && Path.GetExtension(FileUpload1.FileName) == ".xls")
{
using (var excel = new OfficeOpenXml.ExcelPackage(FileUpload1.PostedFile.InputStream))
{
var tbl = new DataTable();
var ws = excel.Workbook.Worksheets.First();
var hasHeader = true; // adjust accordingly
// add DataColumns to DataTable
foreach (var firstRowCell in ws.Cells[1, 1, 1, ws.Dimension.End.Column])
tbl.Columns.Add(hasHeader ? firstRowCell.Text
: String.Format("Column {0}", firstRowCell.Start.Column));
// add DataRows to DataTable
int startRow = hasHeader ? 2 : 1;
for (int rowNum = startRow; rowNum <= ws.Dimension.End.Row; rowNum++)
{
var wsRow = ws.Cells[rowNum, 1, rowNum, ws.Dimension.End.Column];
DataRow row = tbl.NewRow();
foreach (var cell in wsRow)
row[cell.Start.Column - 1] = cell.Text;
tbl.Rows.Add(row);
}
var distinctNames = (from row in tbl.AsEnumerable()
select row.Field<string>("Employee Code")).Distinct();
DataRow[] dataRows = tbl.Select().OrderBy(u => u["Employee Code"]).ToArray();
var ss = dataRows.Where(p => p.Field<string>("Employee Code") == "55").ToArray();
}
}
}
catch (Exception ex) { }
}
The result i'm getting is:
Employee Code Employee Name Date Time In / Out
55 Alex 12/27/2018 8:59 IN
55 Alex 12/27/2018 8:59 IN
55 Alex 12/27/2018 13:00 OUT
55 Alex 12/27/2018 13:00 OUT
55 Alex 12/27/2018 13:48 IN
55 Alex 12/27/2018 13:49 IN
55 Alex 12/27/2018 18:08 OUT
And I want to have first In and then out and then in and then out. This would iterate four times to generate the result.
Expected result is:
Employee Code Employee Name Date Time In / Out
55 Alex 12/27/2018 8:59 IN
55 Alex 12/27/2018 13:00 OUT
55 Alex 12/27/2018 13:48 IN
55 Alex 12/27/2018 18:08 OUT
Can you try to do groupby in the result like below
ss=ss.GroupBy(x=>x.DateTime).ToArray();
Build a logic, if your result have 2 successive In/Out as a sample like below.
Here In I considered as field name
var tt;
for(int i=0;i<ss.Count();i++)
{
if(ss[i].In=="In" && (tt!=null || tt.LastOrDefault().In!="In"))
tt=ss[i];
else if(ss[i].In=="Out" && (tt!=null || tt.LastOrDefault().In!="Out"))
tt=ss[i];
}
I am doing Oracle database project which called "Theatre booking system".
I am trying to display discounted price through Customer_Concession and Member_Concession.
Null-able foreign keys are bookingId, cconcessionId, and mconcessionId in Ticket table.
I want to display all ticket even one of id is not exist.
How can I write the SQL?
Could you help me?
Thanks.
SELECT t.ticketId, pro.name "Production name", PRICE.LEVELID "Price level",
Price.price "Price", (Price.price - ccons.discountPrice - mcons.discountPrice)
"Discounted Price", t.seatNo "Seat", t.rowNo "Row", t.block "Block",
per.performance_date "Performance date", per.start_time "Start time",
per.end_time "End time", t.availability "Availability"
FROM Ticket t, Production pro, Performance per, Price, Price_level,
Booking, Customer, Customer_Concession ccons, Member_concession mcons
WHERE t.performanceid = per.performanceid AND
t.PRODUCTIONID = Price.PRODUCTIONID AND
t.levelId = Price.levelId AND
Price.PRODUCTIONID = pro.PRODUCTIONID AND
Price.levelId = Price_level.levelId AND
t.bookingId = Booking.bookingId AND
Booking.customerId = Customer.customerId AND
ccons.cconcessionId = t.cconcessionId AND
mcons.mconcessionId = t.mconcessionId
ORDER BY t.ticketId
What you are looking for is called LEFT OUTER JOIN.
for more info, visit this site: http://www.oreillynet.com/network/2002/04/23/fulljoin.html
Try...
WHERE t.performanceid = per.performanceid AND
t.PRODUCTIONID = Price.PRODUCTIONID AND
t.levelId = Price.levelId AND
Price.PRODUCTIONID = pro.PRODUCTIONID AND
Price.levelId = Price_level.levelId AND
t.bookingId = Booking.bookingId(+) AND
Booking.customerId = Customer.customerId(+) AND
ccons.cconcessionId(+) = t.cconcessionId AND
mcons.mconcessionId(+) = t.mconcessionId
(note old syntax)