In Hadoop program, I tried to compress the result, I wrote the following code:
FileOutputFormat.setCompressOutput(job, true);
FileOutputFormat.setOutputCompressorClass(job, GzipCodec.class);
The result was compressed, and when I delete the first line:
FileOutputFormat.setCompressOutput(job, true);
and execute the program again, the result was same, was the above code
FileOutputFormat.setCompressOutput(job, true);
optional? What is the function of that code?
Please see the below methods in FileOutPutFormat.java which internally calls the method call which you have deleted.
i.e setCompressOutput(conf, true);
That means you are trying apply Gzip codec class then obviously its a pointer to code that output should be compressed. Isnt it ?
/**
* Set whether the output of the job is compressed.
* #param conf the {#link JobConf} to modify
* #param compress should the output of the job be compressed?
*/
public static void setCompressOutput(JobConf conf, boolean compress) {
conf.setBoolean("mapred.output.compress", compress);
}
/**
* Set the {#link CompressionCodec} to be used to compress job outputs.
* #param conf the {#link JobConf} to modify
* #param codecClass the {#link CompressionCodec} to be used to
* compress the job outputs
*/
public static void
setOutputCompressorClass(JobConf conf,
Class<? extends CompressionCodec> codecClass) {
setCompressOutput(conf, true);
conf.setClass("mapred.output.compression.codec", codecClass,
CompressionCodec.class);
}
Related
Anyone know if a single listener can listens to multiple topic like below? I know just "topic1" works, what if I want to add additional topics? Can you please show example for both below? Thanks for the help!
#KafkaListener(topics = "topic1,topic2")
public void listen(ConsumerRecord<?, ?> record, Acknowledgment ack) {
System.out.println(record);
}
or
ContainerProperties containerProps = new ContainerProperties(new TopicPartitionInitialOffset("topic1, topic2", 0));
Yes, just follow the #KafkaListener JavaDocs:
/**
* The topics for this listener.
* The entries can be 'topic name', 'property-placeholder keys' or 'expressions'.
* Expression must be resolved to the topic name.
* Mutually exclusive with {#link #topicPattern()} and {#link #topicPartitions()}.
* #return the topic names or expressions (SpEL) to listen to.
*/
String[] topics() default {};
/**
* The topic pattern for this listener.
* The entries can be 'topic name', 'property-placeholder keys' or 'expressions'.
* Expression must be resolved to the topic pattern.
* Mutually exclusive with {#link #topics()} and {#link #topicPartitions()}.
* #return the topic pattern or expression (SpEL).
*/
String topicPattern() default "";
/**
* The topicPartitions for this listener.
* Mutually exclusive with {#link #topicPattern()} and {#link #topics()}.
* #return the topic names or expressions (SpEL) to listen to.
*/
TopicPartition[] topicPartitions() default {};
So, your use-case should be like:
#KafkaListener(topics = {"topic1" , "topic2"})
If we have to fetch multiple topics from the application.properties file :
#KafkaListener(topics = { "${spring.kafka.topic1}", "${spring.kafka.topic2}" })
Using Visual Studio Code for JS programming I can access some features from typescript; since the editor will parse all .d.ts files around, it'll help me with variable types. For example it does recognize the following:
any.js
/**
* #param {string} s
* #return {Promise<Person>}
*/
function foo(s){ ... }
foo('Jhon').then((p) => p.name )
index.d.ts
interface Person {
name: string
surname: string
}
Now, I want to access types (interfaces, classes... whatever) declared in node.d.ts declaration file; for example it declares the module stream which declares the Readable interface.
I'm looking for something like this:
const stream = require('stream')
/**
* #param {stream.Readable} stream
*/
function goo(stream) { ... }
But that does not work.I've tried with:
{internal.Readable}
{stream.Readable}
{Node.stream.Readable}
{Node.Readable}
{Node.internal.Readable}
As of VS Code 1.18, this is a limitation when using require with JSDoc types. I've opened this issue to track this specifically
Some possible workarounds:
Use import:
import * as stream from 'stream'
/**
* #param {stream.Readable} stream
*/
function goo(stream) { ... }
or import Readable explicitly:
const stream = require('stream')
const {Readable} = require('stream')
/**
* #param {Readable} stream
*/
function goo(stream) { ... }
https://github.com/Microsoft/TypeScript/issues/14377 also tracks allowing you to specify module imports in jsdocs directly.
I have Laravel setup so Monolog logs to syslog. Here is whats' in my start/global.php:
$monolog = Log::getMonolog();
$monolog->pushHandler(new Monolog\Handler\SyslogHandler('mywebsitename'));
This works well, except all log messages are dumped into /var/log/messages.
How can I specify a specific log file instead of messages? I've seen something called custom "channels" with Monolog. Is it something to do with that?
Also, how do I make sure that this log file is rotated like the rest of my log files?
I'm using Laravel 4.2, and in my setLog function I've added the RotatingFileHandler. This will solve both problems for you.
This code creates a file called mttParser-{Y-m-d}.log in app/storage/logs/import/, where {Y-m-d} is converted to todays date. The second parameter to the RotatingFileHandler, 10, sets to keep 10 versions of the log file before deleting them.
The IntrospectionProcessor simply adds the method, class, line number, where the log was called.
trait LogTrait
{
/**
* #var string The location within the storage path to put the log files
*/
protected $logFileLocation = 'logs/import/';
protected $logFileName = 'mttParser.log';
/**
* #var Logger
*/
protected $log;
/**
* Returns a valid log file, re-using an existing one if possible
*
* #return \Monolog\Logger
*/
public function getLog()
{
// Set a log file
if (!isset( $this->log )) {
$this->setLog();
}
// If it's been set somewhere else, re-create it
if ('Monolog\Logger' !== get_class( $this->log )) {
$this->setLog();
}
return $this->log;
}
/**
* Sets the log file
*/
public function setLog()
{
$this->log = new Logger( 'mttParserLog' );
// Make sure our directory exists
if (!File::isDirectory( storage_path( $this->logFileLocation ) )) {
File::makeDirectory( storage_path( $this->logFileLocation ) );
}
$this->log->pushHandler( new RotatingFileHandler( storage_path( $this->logFileLocation . $this->logFileName ),
10 ) );
$this->log->pushProcessor( new IntrospectionProcessor() );
}
}
I am constructing a tree using the Google Closure Library. Now I want the nodes to expand on a single mouseclick, but I seem not to get it working.
I've tried calling goog.ui.component.EventType.SELECT, but it won't work.
At my tree-component class I've added the following event:
goog.events.listen(item, [goog.ui.Component.EventType.SELECT, goog.ui.tree.BaseNode.EventType.EXPAND], this.dispatchEvent, false, this);
And at my class calling the function i've added:
goog.events.listen(this._tree, [goog.ui.Component.EventType.SELECT, goog.ui.tree.BaseNode.EventType.EXPAND], this.treeClick, false, this)
Any suggestions on how I could expand my node with a single click?
I see that BaseNode is screwing up any click events:
/**
* Handles a click event.
* #param {!goog.events.BrowserEvent} e The browser event.
* #protected
* #suppress {underscore}
*/
goog.ui.tree.BaseNode.prototype.onClick_ = goog.events.Event.preventDefault;
When adding this code to the goog/demos/tree/demo.html:
goog.ui.tree.BaseNode.prototype.onClick_ = function (e) {
var qq = this.expanded_?this.collapse():this.expand();
};
The tree control expands and collapses on one click.
Tried to extend goog.ui.tree.TreeControl and override createNode to return a custom goog.ui.tree.TreeNode that overrides onClick but could not do it without getting errors. In theory you could create your custom TreeControl that expands and collapses on one click.
If you want to support non collapsable content and other features I guess you have to trigger some sort of event instead of just extend or callapse the TreeNode.
[update]
After some fiddling I got it working by subclassing TreeControl and TreeNode added the following code to goog/demos/tree/demo.html
/**
* This creates a myTreeControl object. A tree control provides a way to
* view a hierarchical set of data.
* #param {string} html The HTML content of the node label.
* #param {Object=} opt_config The configuration for the tree. See
* goog.ui.tree.TreeControl.defaultConfig. If not specified, a default config
* will be used.
* #param {goog.dom.DomHelper=} opt_domHelper Optional DOM helper.
* #constructor
* #extends {goog.ui.tree.BaseNode}
*/
var myTreeControl = function (html, opt_config, opt_domHelper) {
goog.ui.tree.TreeControl.call(this, html, opt_config, opt_domHelper);
};
goog.inherits(myTreeControl, goog.ui.tree.TreeControl);
/**
* Creates a new myTreeNode using the same config as the root.
* myTreeNode responds on single clicks
* #param {string} html The html content of the node label.
* #return {goog.ui.tree.TreeNode} The new item.
* #override
*/
myTreeControl.prototype.createNode = function (html) {
return new myTreeNode(html || '', this.getConfig(),
this.getDomHelper());
};
/**
* A single node in the myTreeControl.
* #param {string} html The html content of the node label.
* #param {Object=} opt_config The configuration for the tree. See
* goog.ui.tree.TreeControl.defaultConfig. If not specified, a default config
* will be used.
* #param {goog.dom.DomHelper=} opt_domHelper Optional DOM helper.
* #constructor
* #extends {goog.ui.tree.BaseNode}
*/
var myTreeNode = function (html, opt_config, opt_domHelper) {
goog.ui.tree.TreeNode.call(this,html, opt_config, opt_domHelper)
}
goog.inherits(myTreeNode, goog.ui.tree.TreeNode);
/**
* Handles a click event.
* #param {!goog.events.BrowserEvent} e The browser event.
* #override
*/
myTreeNode.prototype.onClick_ = function (e) {
goog.base(this, "onClick_", e);
this.onDoubleClick_(e);
};
Changed the creation of the tree variable:
tree = new myTreeControl('root', treeConfig);
Works on single clicks and have not noticed any other things breaking. I've added the annotations so it'll compile easier. You might put the declarations in a separate file with a goog.provide.
Too bad the handleMouseEvent_ of TreeControl is private or you'll just override that one but you can't without changing either TreeControl source or getting compile errors/warnings. Ach wel, jammer.
I received the following error:
Call to undefined method CI_Loader::plugin() in C:\wamp\www\Code\application\libraries\DX_Auth.php on line 1233
on this code:
function captcha()
{
$this->ci->load->helper('url');
$this->ci->load->plugin('dx_captcha');
$captcha_dir = trim($this->ci->config->item('DX_captcha_path'), './');
Make sure you have moved any array values in application/config/autoload.php from $autoload[‘plugins’] to $autoload[‘helpers’] or you will notice stuff break.
This is the reference
Which version of CI are you using? Plugins has been removed since the 2.x and replaced with helper.
Try to use reCaptcha instead, it has a good library.
You're trying to load a plugin and plugins are not supported, if I remember it right since CI version 2. If that's the case (which seems to be) you need to convert your plugins into helpers.
I think you are trying to use an old version of DX_Auth on new version of CodeIgniter. Current version of DX_Auth is compatible with CI 2.x and is available on github.
A simple way to solve this problem is that you just put this code in your loader.php. The plugin its works. Go to System->Core->Loader.php.
/**
* Load Plugin
*
* This function loads the specified plugin.
*
* #access public
* #param array
* #return void
*/
function plugin($plugins = array())
{
if ( ! is_array($plugins))
{
$plugins = array($plugins);
}
foreach ($plugins as $plugin)
{
$plugin = strtolower(str_replace(EXT, '', str_replace('_pi', '', $plugin)).'_pi');
if (isset($this->_ci_plugins[$plugin]))
{
continue;
}
if (file_exists(APPPATH.'plugins/'.$plugin.EXT))
{
include_once(APPPATH.'plugins/'.$plugin.EXT);
}
else
{
if (file_exists(BASEPATH.'plugins/'.$plugin.EXT))
{
include_once(BASEPATH.'plugins/'.$plugin.EXT);
}
else
{
show_error('Unable to load the requested file: plugins/'.$plugin.EXT);
}
}
$this->_ci_plugins[$plugin] = TRUE;
log_message('debug', 'Plugin loaded: '.$plugin);
}
}
// --------------------------------------------------------------------
/**
* Load Plugins
*
* This is simply an alias to the above function in case the
* user has written the plural form of this function.
*
* #access public
* #param array
* #return void
*/
function plugins($plugins = array())
{
$this->plugin($plugins);
}