I am tying to do a simple map reduce function in Hadoop 2.3.0 using the org.apache.hadoop.mapreduce API and yet when I try to to run it I get the following error
org.apache.hadoop.mapreduce.lib.input.FileSplit cannot be cast to org.apache.hadoop.mapred.InputSplit.
Since what I call in the mapreduce function I am using is this
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.conf.*;
import org.apache.hadoop.io.*;
import org.apache.hadoop.mapreduce.*;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.input.TextInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat;
I have no idea why it keeps using the old API, is there some configuration file I have modify?
Try using hadoop dfsadmin -refreshNodes command to refesh the nodes. So that it will fetch new API.
Related
My bot.py 's code is :
import discord
from discord.ext import commands
import random
from discord.ext.commands import bot
import asyncio
import requests
import os
client = commands.Bot(command_prefix = "!")
.. Command's and stuff..
bot.run(str(os.environ.get('DISCORD_TOKEN')))
My requirements.txt consists of :
discord.py
requests
My Procfile consists of :
worker: python3 bot.py
I followed all steps to deploy through git and it successfully got deployed but still it's not online.
You bot doesn't go online because of your os.environ. It doesn't have any get method since it's a Mapping object. Here's how you use it:
bot.run(os.environ['DISCORD_TOKEN'])
PS: you have some unecessary imports, like from discord.ext.commands import bot. You should only import what you need (eg. from os import environ instead of import os).
I'm getting this error on the airflow UI. the code line I've entered is:
from airflow.contrib.sensors.file_sensor import FileSensor
any ideas?
found nothing on google.
thanks
Are you using an Airflow version < 1.10, e.g. 1.9?
In that case, try the following:
from airflow.contrib.operators.fs_operator import FileSensor
I tried this in v1.10.9 and the following both work.
from airflow.contrib.sensors.file_sensor import FileSensor
from airflow.contrib.operators.fs_operator import FileSensor
See the following links for reference:
https://github.com/apache/airflow/blob/v1-9-stable/airflow/contrib/operators/fs_operator.py
https://github.com/apache/airflow/blob/v1-9-stable/airflow/contrib/operators/init.py
I am trying to create a very simple Katalon test case that opens Firefox, goes to given URL and clicks a button to download a file. I have set up Desired Capabilities according to the Katalon documentation (https://github.com/katalon-studio/docs/blob/master/pages/katalon-studio/docs/introduction-to-desired-capabilities.md) but with no luck. When I try to download a file prompt shows up and file is not downloaded. How can I disable the prompt and download the file immediately instead?
Software versions, source code and screenshots below.
Windows 10, Katalon Studio 7.2.1, Mozilla Firefox 72.0.2, Selenium 3.141.59
import static com.kms.katalon.core.checkpoint.CheckpointFactory.findCheckpoint
import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase
import static com.kms.katalon.core.testdata.TestDataFactory.findTestData
import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject
import static com.kms.katalon.core.testobject.ObjectRepository.findWindowsObject
import com.kms.katalon.core.checkpoint.Checkpoint as Checkpoint
import com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords as CucumberKW
import com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords as Mobile
import com.kms.katalon.core.model.FailureHandling as FailureHandling
import com.kms.katalon.core.testcase.TestCase as TestCase
import com.kms.katalon.core.testdata.TestData as TestData
import com.kms.katalon.core.testobject.TestObject as TestObject
import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
import com.kms.katalon.core.windows.keyword.WindowsBuiltinKeywords as Windows
import internal.GlobalVariable as GlobalVariable
import org.openqa.selenium.Keys as Keys
WebUI.openBrowser('https://file-examples.com/index.php/text-files-and-archives-download/')
WebUI.click(findTestObject('downloadCsvFileButton'))
For Chrome the default setup of Desired Capabilities in Project Settings works fine, but for Firefox I had to do some workaround to make it work.
So, I found this topic https://forum.katalon.com/t/opening-firefox-with-a-specific-non-anonymous-profile/12012/15 and #kazurayam 's reply helped me to create a script that initializes WebDriver which I call before each test case:
import org.openqa.selenium.WebDriver
import org.openqa.selenium.firefox.FirefoxDriver
import org.openqa.selenium.firefox.FirefoxOptions
import org.openqa.selenium.firefox.FirefoxProfile
import org.openqa.selenium.firefox.ProfilesIni
import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject
import com.kms.katalon.core.webui.driver.DriverFactory
import com.kms.katalon.core.webui.driver.WebUIDriverType
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
import internal.GlobalVariable as GlobalVariable
WebUIDriverType executedBrowser = DriverFactory.getExecutedBrowser()
switch(executedBrowser) {
case WebUIDriverType.FIREFOX_DRIVER: // "Firefox"
System.setProperty('webdriver.gecko.driver', DriverFactory.getGeckoDriverPath())
FirefoxOptions options = new FirefoxOptions()
options.addPreference('marionette', true)
options.addPreference('browser.download.folderList', 2)
options.addPreference('browser.helperApps.alwaysAsk.force', false)
options.addPreference('browser.download.manager.showWhenStarting', false)
options.addPreference('browser.download.dir', GlobalVariable.downloadPath)
options.addPreference('browser.download.downloadDir', GlobalVariable.downloadPath)
options.addPreference('browser.download.defaultFolder', GlobalVariable.downloadPath)
options.addPreference('browser.helperApps.neverAsk.saveToDisk', 'application/download, application/octet-stream, text/csv')
WebDriver driver = new FirefoxDriver(options);
// let Katalon Studio to use the WebDriver created here
DriverFactory.changeWebDriver(driver)
break
default:
WebUI.openBrowser('')
}
Note to others, if you want to download different file types you have to specify all the required the MIME types in ‘browser.helperApps.neverAsk.saveToDisk’ preference. A list of MIME types can be found here:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Complete_list_of_MIME_types
Additionally, if the file is PDF you have to add one more preference:
options.addPreference('pdfjs.disabled', true)
I have created a sample application using dotnet core, I have also implemented web-pack for bundling angular.
My problem is the bundled file is really big, around 6MB.
Here is my polyfills.ts file
import 'ie-shim'; // Internet Explorer 9 support.
import 'core-js/es6/symbol';
import 'core-js/es6/object';
import 'core-js/es6/function';
import 'core-js/es6/parse-int';
import 'core-js/es6/parse-float';
import 'core-js/es6/number';
import 'core-js/es6/math';
import 'core-js/es6/string';
import 'core-js/es6/date';
import 'core-js/es6/array';
import 'core-js/es6/regexp';
import 'core-js/es6/map';
import 'core-js/es6/set';
import 'core-js/es6/weak-map';
import 'core-js/es6/weak-set';
import 'core-js/es6/typed';
import 'core-js/es6/reflect';
import 'core-js/es7/reflect';
import 'ts-helpers';
Here my is network status
How can I reduce the file size of the bundled file?
<my-app>Loading AppComponent content here ...</my-app> This text is showing before the component is visible, how can I speed up it?
If you are using the webpack, you need to use the command parameter p for production.
webpack -p --progress
As a newby to servicemix/karaf, I'm trying to create a very simple program that accepts and returns a REST-request. The class I have is:
package (....)
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.ResponseBody;
import org.springframework.web.bind.annotation.RestController;
import (...).model.RSDocument;
import (...).model.RSDocumentResponse;
#RestController
public class DocumentService {
#RequestMapping(value = "/rest/document", method = RequestMethod.POST,
produces = MediaType.APPLICATION_JSON_VALUE)
public #ResponseBody ResponseEntity<RSDocumentResponse> printDocument(
#RequestBody RSDocument documentRequest)
{
System.out.println(documentRequest.getContent());
RSDocumentResponse response = new RSDocumentResponse();
response.setSuccess(true);
return new ResponseEntity<>(response, HttpStatus.OK);
}
}
I've got something similar working in Tomcat. In Tomcat, you would specify the port where it will listen to incoming requests by doubleclicking on the server in eclipse and setting a value in the "Ports" section. How do I set the port in Servicemix, or even figure out which port it's currently listening to? I have the bundle succesfully starting from the command line in Servicemix. My application doesn't seem to be listening on 80 (Apache), 8080 (None), or 8181 (Servicemix console)
First of all make sure you have the web-container deployed. For this make sure the war feature is installed.
feature:list | grep war
if it's not installed, install it by issueing:
feature:install war
Now, make sure your jar/war contains the Web-ContextPath Manifest entry telling the web-container which context path to look for. If you have all other required bundles installed and running, including all dependencies are set you should be able to navigate to your rest service on:
localhost:8181/myContextPath/rest/document