Why MQL4 OrderModify() will not modify the order when backtesting? - algorithmic-trading

I'm trying to ADD a stop loss to my open market orders in MetaTrader 4 when a position gets 100 pips "to the good" which is to be equal to the Order Open Price;
OrderStopLoss() == OrderOpenPrice()
But this isn't happening.
I've added Print() & GetLastError() functions and nothing is coming up in the journal, so it must be something in my coding - but cannot see what would be wrong.
OK this is what I have so far, one for loop for the buy, one for the sell. I've also Normalized the "doubles" as I have been advised to do & have also declared the BuyMod & SellMod to "true" at the very top. This should ensure that the default won't resort to false. I also thought it might be helpful if I told you I have the MetaEditor version 5 build 1241:)
The following code I have is the following;
/*Breakeven Order Modification*/
bool BuyMod = true;
bool SellMod = true;
for(int b = OrdersTotal()-1;b>=0;b--)
{
if(OrderSelect(b,SELECT_BY_POS,MODE_TRADES))
{
double aBidPrice = MarketInfo(Symbol(),MODE_BID);
double anOpenPrice = OrderOpenPrice();
double aNewTpPrice = OrderTakeProfit();
double aCurrentSL = OrderStopLoss();
double aNewSLPrice = anOpenPrice;
double pnlPoints = (aBidPrice - anOpenPrice)/_Point;
double stopPoints = (aBidPrice - aNewSLPrice)/_Point;
int stopLevel = int(MarketInfo(Symbol(),MODE_STOPLEVEL));
int aTicket = OrderTicket();
if(OrderType() == OP_BUY)
if(stopPoints >= stopLevel)
if(aTicket > 0)
if(pnlPoints >= breakeven)
if(aNewSLPrice != aCurrentSL)
{
BuyMod = OrderModify(OrderTicket(),OrderOpenPrice(),NormalizeDouble(aNewSLPrice,Digits),NormalizeDouble(aNewTpPrice,Digits),0,buycolor);
SendMail("Notification of Order Modification for Ticket#"+IntegerToString(OrderTicket(),10),"Good news! Order Ticket#"+IntegerToString(OrderTicket(),10)+"has been changed to breakeven");
}
}
}
for(int s = OrdersTotal()-1; s>=0; s--)
{
if(OrderSelect(s,SELECT_BY_POS,MODE_TRADES))
{
double anAskPrice = MarketInfo(Symbol(),MODE_ASK);
double anOpenPrice = OrderOpenPrice();
double aNewTpPrice = OrderTakeProfit();
double aCurrentSL = OrderStopLoss();
double aNewSLPrice = anOpenPrice;
double pnlPoints = (anOpenPrice - anAskPrice)/_Point;
double stopPoints = (aNewSLPrice - anAskPrice)/_Point;
int stopLevel = int(MarketInfo(Symbol(),MODE_STOPLEVEL));
int aTicket = OrderTicket();
if(OrderType()== OP_SELL)
if(stopPoints >= stopLevel)
if(pnlPoints >= breakeven)
if(aNewSLPrice != aCurrentSL)
if(aTicket > 0)
{
SellMod = OrderModify(OrderTicket(),OrderOpenPrice(),NormalizeDouble(aNewSLPrice,Digits),NormalizeDouble(aNewTpPrice,Digits),0,sellcolor);
SendMail("Notification of Order Modification for Ticket#"+IntegerToString(OrderTicket(),10),"Good news! Order Ticket#"+IntegerToString(OrderTicket(),10)+"has been changed to breakeven");
}
}
}
trading algorithmic-trading mql4 metatrader4
shareeditdeleteflag
edited just now
asked 2 days ago
Todd Gilbey
264
You might want to know, StackOverflow does not promote duplicate questions. ( see the

Besides meeting an MQL4 syntax-rules,there are more conditions:
A first hidden trouble is in number rounding issues.
MetaQuotes, Inc., recommends wherever possible, to normalise float values into a proper price-representation.
Thus,wherever a price goes into a server-side instruction { OrderSend(), OrderModify(), ... } one shall always prepare such aPriceDOMAIN valueby a call to NormalizeDouble( ... , _Digits ), before a normalised price hits any server-side instruction call.
May sound rather naive, but this saves you issues with server-side rejections.
Add NormalizeDouble() calls into your code on a regular base as your life-saving vest.
A second, even a better hidden trouble is in STOP_ZONE-s and FREEZE_ZONE-s
While not visible directly, any Broker set's in their respective Terms & Conditions these parameters.
In practice,this means, if you instruct { OrderSend() | OrderModify() } to set / move aPriceDOMAIN level to be setup too close to current actual Ask/Bid ( violating a Broker-forbidden STOP_ZONE )orto delete / modify aPriceDOMAIN level of TP or SL, that are already set and is right now, within a Broker-forbidden FREEZE_ZONE distance from actual Ask/Bid,such instruction will not be successfully accepted and executed.
So besides calls to the NormalizeDouble(), always wait a bit longer as the price moves "far" enough and regularly check for not violating forbidden STOP_ + FREEZE_ zones before ordering any modifications in your order-management part of your algotrading projects.
Anyway, Welcome to Wild Worlds of MQL4
Update: while StackOverflow is not a Do-a-Homework site, let me propose a few directions for the solution:
for ( int b = OrdersTotal() - 1; b >= 0; b-- ) // ________________________ // I AM NOT A FAN OF db.Pool-looping, but will keep original approach for context purposes
{ if ( ( OrderSelect( b, SELECT_BY_POS, MODE_TRADES ) ) == true )
{ // YES, HAVE TO OPEN A CODE-BLOCK FOR if()-POSITIVE CASE:
// ------------------------------------------------------
double aBidPRICE = MarketInfo( Symbol(), MODE_BID ); // .UPD
double anOpenPRICE = OrderOpenPrice(); // .SET FROM a db.Pool Current Record
double aNewTpPRICE = OrderTakeProfit(); // .SET FROM a db.Pool Current Record
double aCurrentSlPRICE = OrderStopLoss(); // .SET FROM a db.Pool Current Record
double aNewSlPRICE = anOpenPRICE; // .SET
double pnlPOINTs = ( aBidPRICE - anOpenPRICE )/_Point; // .SET
double stopPOINTs = ( aBidPRICE - aNewSlPRICE )/_Point; // .SET
// ------------------------------------------------------------ // .TEST
if ( OP_BUY == OrderType() )
if ( Period() == OrderMagicNumber() )
if ( stopPOINTa > stopLevel )
if ( pnlPOINTs >= breakeven )
if ( aNewSlPRICE != aCurrentSlPRICE )
{ // YES, HAVE TO OPEN A BLOCK {...}-CODE-BLOCK FOR THE if()if()if()if()-chain's-POSITIVE CASE:
// -------------------------------------------------------------------------------------------
int aBuyMOD = OrderModify( OrderTicket(),
OrderOpenPrice(),
NormalizeDouble( aNewSlPRICE, Digits ),
NormalizeDouble( aNewTpPRICE, Digits ),
0,
buycolor
);
switch( aBuyMOD )
{ case ( NULL ): { ...; break; } // FAIL ( ANALYSE ERROR )
default: { ...; break; } // PASS OrderModify()
}
}
}

The problem is in your call to a built-in OrderModify() function.
OrderStopLoss() == OrderModify() will evaluate as false which in turn will evaluate as 0 since == is a comparison operator.
An OrderStopLoss() is a call to another built-in function (not a variable), you can't save anything to it so OrderStopLoss() = 4 wouldn't work either.
From the MQL4 documentation:
bool OrderModify( int ticket, // ticket
double price, // price
double stoploss, // stop loss
double takeprofit, // take profit
datetime expiration, // expiration
color arrow_color // color
);
In your case that would be the following, assuming ModBuy is already defined somewhere in the code:
ModBuy = OrderModify( OrderTicket(), // <-ticket from record OrderSelect()'d
OrderOpenPrice(), // <-price from current record
OrderOpenPrice(), // <-price from current record
OrderTakeProfit(), // <-TP from current record
0, // ( cannot set P/O expiration for M/O )
buycolor // ( set a color for a GUI marker )
);
Or you could just use any other valid value instead of the second OrderOpenPrice() to set a new stoploss.

I'm really sorry, I'm new to Stackoverflow, this is the revised code I now have based on everyone's comments & recommendation's below
**Local Declarations**
pnlPoints = 0;
point = MarketInfo(Symbol(),MODE_POINT);
stopLevel = int(MarketInfo(Symbol(),MODE_STOPLEVEL)+MarketInfo (Symbol(),MODE_SPREAD));
sl = NormalizeDouble(OrderStopLoss(),Digits);
tp = OrderTakeProfit();
cmd = OrderType();
breakeven = 100;
**Global Variables**
double pnlPoints;
double price,sl,tp;
double point;
int stopLevel;
int cmd;
int breakeven;
double newSL;
for(int b = OrdersTotal()-1; b>=0; b--)
{
if((OrderSelect(b,SELECT_BY_POS,MODE_TRADES))==true)
price = MarketInfo(Symbol(),MODE_BID);
newSL = NormalizeDouble(OrderOpenPrice(),Digits);
pnlPoints = (price - OrderOpenPrice())/point;
{
if(OrderType()==OP_BUY)
if(OrderMagicNumber() == Period())
if((price-newSL)/point>=stopLevel)
if(pnlPoints>=breakeven)
if(sl!=newSL)
ModBuy = OrderModify(OrderTicket(),OrderOpenPrice(),newSL,tp,buycolor);
else if(ModBuy == false)
{
Print("OrderModify failed with error #",GetLastError());
}
}
}

Related

How to modify a StopLoss of an active trade?

I have trouble with modifying the stoploss of a running trade using MQL5. Selecting the order works out for me. But if I try to access the variables ( for instance OrderTicket() & OrderOpenPrice() ), it always returns 0.00000:
2017.06.01 00:06:32.114 2016.04.08 00:00:00 failed modify buy 0.00 sl: 0.00000, tp: 0.00000 -> sl: 1.41594, tp: 0.00000 [Invalid request]
Here's my stoploss modyfing void:
void modifyStops() {
int total = OrdersTotal(); // total number of placed pending orders
Print( total + " Orders on the line!!!" );
//--- Over all placed pending orders
for ( int i = 0; i < total; i++ )
{ bool isOrderSelected = OrderSelect( i, SELECT_BY_POS, MODE_TRADES );
if ( isOrderSelected )
{ // TODO: Check the Trades to contain the correct Order Number
Print( "Symbol & Magicnumber matching" );
double newStopLoss;
// Update the stop loss
if ( OrderType() == OP_BUY )
{
newStopLoss = addTolerance( SL );
}
else if ( OrderType() == OP_SELL )
{
newStopLoss = minusTolerance( SL );
}
newStopLoss = NormalizeDouble( newStopLoss, Digits );
Print( "NEW STOP LOSS::::=====> ", Symbol(), " at ", newStopLoss );
if ( !OrderModify( OrderTicket(), OrderOpenPrice(), newStopLoss, OrderTakeProfit(), 0, Green ) )
{
Print( "OrderModify returned the error of ", GetLastError() );
}
}
}
The CTrade class isn't working properly for me. I tried to implement the code you've posted - however: It still doesn't seem to work out.
Unfortunately I implemented that in my EA and the OrderGetTicket(i) returns zero when the trade is live. So my void looks like this:
void modifyStops() {
for(int i=0; i<PositionsTotal();i++)
{
ulong ticket;
if((ticket=PositionGetTicket(i))>0)
{
//--- return order properties
double open_price =PositionGetDouble(POSITION_PRICE_OPEN);
datetime time_open =(datetime)PositionGetInteger(POSITION_TIME);
string symbol =PositionGetString(POSITION_SYMBOL);
int order_magic =PositionGetInteger(POSITION_MAGIC);
double volume =PositionGetDouble(POSITION_VOLUME);
double stoploss =PositionGetDouble(POSITION_SL);
double takeprofit =PositionGetDouble(POSITION_TP);
ENUM_ORDER_TYPE type =EnumToString(ENUM_ORDER_TYPE(PositionGetInteger(POSITION_TYPE)));
//--- prepare and show information about the order
printf("#ticket %d %s %G %s at %G, with sl: %G tp: %G was set up at %s",
ticket, // order ticket
type, // type
volume, // placed volume
symbol, // symbol
open_price, // specified open price
stoploss, //
takeprofit, //
TimeToString(time_open) // time of order placing
);
}
}
}
And the printf function returns nothing:
2017.06.02 01:42:26.910 2016.04.07 00:00:00 #ticket 1 (non-string passed) 0 at 0, with sl: 0 tp: 0 was set up at 1970.01.01 00:00
I can't believe it's that hard to simply modify an SL in MQL5. That's horribly. However I need to get through it to test my strategy over several pairs...
Do you have another idea? I set up trades with the following code:
void createPendingOrder(ENUM_ORDER_TYPE orderType, double lots, double stopLoss) {
MqlTradeRequest request={0};
MqlTradeResult result={0};
//--- parameters to place a pending order
request.action =TRADE_ACTION_PENDING; // type of trade operation
request.symbol =Symbol(); // symbol
request.volume =lots; // volume of 0.1 lot
//request.deviation=2; // allowed deviation from the price
request.magic =224466 ; // MagicNumber of the order
//int offset = 3; // offset from the current price to place the order, in points
double price; // order triggering price
double point=SymbolInfoDouble(_Symbol,SYMBOL_POINT); // value of point
int digits=SymbolInfoInteger(_Symbol,SYMBOL_DIGITS); // number of decimal places (precision)
//--- checking the type of operation
if(orderType==ORDER_TYPE_BUY_STOP)
{
request.type =ORDER_TYPE_BUY_STOP; // order type
price =entryPrice;
request.price =NormalizeDouble(price,digits); // normalized opening price
request.sl =stopLoss;
}
else if(orderType==ORDER_TYPE_SELL_STOP)
{
request.type =ORDER_TYPE_SELL_STOP; // order type
price =entryPrice;
request.price =NormalizeDouble(price,digits); // normalized opening price
request.sl =stopLoss;
}
else Alert("This example is only for placing pending orders"); // if not pending order is selected
//--- send the request
if(!OrderSend(request,result))
PrintFormat("OrderSend error %d",GetLastError()); // if unable to send the request, output the error code
//--- information about the operation
PrintFormat("retcode=%u deal=%I64u order=%I64u",result.retcode,result.deal,result.order);
}
Would it for instance be possible to save the result object in a Array and then access the running trade through that object?
Your problem is you are trying to run MQL4 code in MQL5.
There are no OrderModify(), OrderOpenPrice(), OrderTicket() in MQL5!!!
See the documentation here on how to select, query values and modify trades.
you will need to be using OrderGetDouble(), OrderGetInteger() and OrderGetString() to query open price, stop loss etc.
e.g.
if((ticket=OrderGetTicket(i))>0)
{
//--- return order properties
open_price =OrderGetDouble(ORDER_PRICE_OPEN);
time_setup =(datetime)OrderGetInteger(ORDER_TIME_SETUP);
symbol =OrderGetString(ORDER_SYMBOL);
order_magic =OrderGetInteger(ORDER_MAGIC);
positionID =OrderGetInteger(ORDER_POSITION_ID);
initial_volume=OrderGetDouble(ORDER_VOLUME_INITIAL);
type =EnumToString(ENUM_ORDER_TYPE(OrderGetInteger(ORDER_TYPE)));
//--- prepare and show information about the order
printf("#ticket %d %s %G %s at %G was set up at %s",
ticket, // order ticket
type, // type
initial_volume, // placed volume
symbol, // symbol
open_price, // specified open price
TimeToString(time_setup)// time of order placing
);
}
Orders are modified using the OrderSend() function https://www.mql5.com/en/docs/trading/ordersend
Update
MQL5 uses a much more complex system of Orders, Positions, Deals and historyOrders. An MQL5 community article attempts to explain how they all relate to one another.
Orders = Pending trades (Buy Stop, Buy Limit, Sell Stop, Sell Limit)
Positions = Open trades (Buy, Sell)
HistoryOrders = Closed/Deleted Trades
Deals = transactions that make up an order/position
To loop through and look at pending Orders:
for(int i=0; i<OrdersTotal();i++)
{
if((ticket=OrderGetTicket(i))>0)
{
//--- return order properties
open_price =OrderGetDouble(ORDER_PRICE_OPEN);
time_setup =(datetime)OrderGetInteger(ORDER_TIME_SETUP);
symbol =OrderGetString(ORDER_SYMBOL);
order_magic =OrderGetInteger(ORDER_MAGIC);
positionID =OrderGetInteger(ORDER_POSITION_ID);
initial_volume=OrderGetDouble(ORDER_VOLUME_INITIAL);
type =EnumToString(ENUM_ORDER_TYPE(OrderGetInteger(ORDER_TYPE)));
//--- prepare and show information about the order
printf("#ticket %d %s %G %s at %G was set up at %s",
ticket, // order ticket
type, // type
initial_volume, // placed volume
symbol, // symbol
open_price, // specified open price
TimeToString(time_setup)// time of order placing
);
}
}
To loop through and look at open trades:
for(int i=0; i<PositionsTotal();i++)
{
if((ticket= PositionGetTicket(i))>0)
{
//--- return order properties
open_price =PositionGetDouble(POSITION_PRICE_OPEN);
time_open =(datetime)PositionGetInteger(POSITION_TIME);
symbol =PositionGetString(POSITION_SYMBOL);
order_magic =PositionGetInteger(POSITION_MAGIC);
volume =PositionGetDouble(POSITION_VOLUME);
stoploss =PositionGetDouble(POSITION_SL);
takeprofit =PositionGetDouble(POSITION_TP);
type =EnumToString(ENUM_ORDER_TYPE(PositionGetInteger(POSITION_TYPE)));
//--- prepare and show information about the order
printf("#ticket %d %s %G %s at %G, with sl: %G tp: %G was set up at %s",
ticket, // order ticket
type, // type
volume, // placed volume
symbol, // symbol
open_price, // specified open price
stoploss, //
takeprofit, //
TimeToString(time_open) // time of order placing
);
}
}
Hopefully, someone else will be able to provide a better explanation of Orders, Positions, Deals, history Orders as they still give me a headache.
To make it simpler I usually just create an instance of the CTrade Class
Update2
Minimal example of trailSL for buy Positions using standard trade functions and Ctrade class
Init
Ctrade *m_trade;
CSymbolInfo *m_symbol;
Void OnInit()
{
m_trade = new Ctrade();
m_trade.SetExpertMagicNumber(100);
m_symbol = new CSymbolInfo();
m_symbol.Name(Symbol());
}
void OnTick()
{
m_symbol.RefreshRates();
}
Trail SL of Buy trade with standard functions
void modifysl()
{
ulong ticket;
MqlTradeRequest request = {0};
MqlTradeResult result = {0};
double newsl;
for(int i=0; i<PositionsTotal();i++)
{
ticket=PositionGetTicket(i);
if(ticket>0)
{
request.action = TRADE_ACTION_SLTP; // type of trade operation
request.position = ticket; // ticket of the position
request.symbol = PositionGetString(POSITION_SYMBOL); // symbol
request.sl = PositionGetDouble(POSITION_SL); // Stop Loss of the position
request.tp = PositionGetDouble(POSITION_TP); // Take Profit of the position
request.magic = 100; // MagicNumber of the position
newsl = NormalizeDouble(m_symbol.Bid()-100*m_symbol.Point(),
m_symbol.Digits());
if(newsl>request.sl)
{
request.sl = newsl;
if(!OrderSend(request,result))
{
PrintFormat("OrderSend error %d",GetLastError()); // if unable to send the request, output the error code
}
//--- information about the operation
PrintFormat("retcode=%u deal=%I64u order=%I64u",result.retcode,result.deal,result.order);
}
}
}
}
Trail Buy Position StopLoss using CTrade
void modifyslCtrade()
{
ulong ticket;
MqlTradeRequest request= {0};
MqlTradeResult response ={0};
double newsl;
for(int i=0; i<PositionsTotal();i++)
{
ticket=PositionGetTicket(i);
if(ticket>0)
{
newsl = NormalizeDouble(m_symbol.Bid()-100*m_symbol.Point(),
m_symbol.Digits());
if(newsl>PositionGetDouble(POSITION_SL))
{
m_trade.PositionModify(ticket,
newsl,
PositionGetDouble(POSITION_TP));
}
}
}
}

How to get OHLC-values from each new candle?

I am new in MQL5 and I am trying to capture the values of Open, High, Low and Close of each new candle.
For now I am using a one minute TimeFRAME for each candle.
I read the documentation and have not figured out how can I do it.
My only clue was the CopyOpen() functions, but I am still stuck.
Let's split the task:
How to read OHLC-values?
How to detect (each) new candle?
A1: MQL4/MQL5 syntax reports OHLCV-values straight in Open[], High[], Low[], Close[], Volume[] time-series arrays. As a rule of thumb, these arrays are time-series, reverse-stepping indexed, so that the most recent cell ( The Current Bar ( candle ) ) always has cell-index == 0. So Open[1], High[1], Low[1], Close[1] are values for the "current instrument" ( _Symbol ), retrieved from the "current TimeFRAME" for a candle, that was already closed right before the "current Candle" has started. Complex? Well, just on the first few reads. You will get acquainted with this.
If your code does not want to rely on "current" implicit contexts, the syntax allows one to use explicit, indirect, specifications:
/* iVolume( ||| ... )
iTime( ||| ... )
iClose( ||| ... )
iLow( ||| ... )
iHigh( vvv ... ) */
iOpen( aTradingSymbolNameSTRING, // Broker specific names, "DE-30.." may surprise
PERIOD_M1, // explicit reference to use M1 TimeFRAME
1 // a Cell-index [1] last, closed Candle
)
A2: There is neat way how to detect a new Candle, indirectly, the same trick allows one to thus detect a moment, when the previous Candle stops evolving ( values do not change anymore ) which thus makes sense to report "already frozen" OHLCV-values to be reported anywhere else.
Remeber, the "current" OHLCV-registers-[0] are always "hot" == continuously changing throughout the time of the "current" TimeFRAME Candle duration, so one has to wait till a new Candle starts ( indirectly meaning the "now-previous" Candle [0] has ended and has thus got a reverse-stepping index "re-indexed" to become [1], a frozen one ).
For detecting a new candle it is enough to monitor changes of a system register int Bars, resp. an indirect, context aware, int iBars( ... ).
One may realise, that there are some "theoretical" Candles, that do not happen and are thus not "visible" / "accessible" in data of time-series -- whence a market was not active during such period of time and no PriceDOMAIN change has happened during such administratively-framed epoch in time -- for such situations, as there was no price-change, there was no QUOTE and thus such candle did not happen and is "missing" both in linear counting and in data-cells. The first next QUOTE arrival is thus painted right "besides" a candle, that was principally "older" than a "previous"-neighbour ( the missing candles are not depicted, so due care ought be taken in processing ). This typically happens even on major instruments near the Friday EoB/EoWk market closing times and around midnights UTC +0000 during the 24/5-cycles.
In case you become too frustrated, here is a script that will export selected chart contents the second a new candle appears. Just choose the pair you want and attach this to the chart and you will get exported a .csv file on each new candle.
//+------------------------------------------------------------------+
#include <stdlib.mqh>
#include <stderror.mqh>
//+------------------------------------------------------------------+
//| Input Parameters Definition |
//+------------------------------------------------------------------+
extern int BarCount = 500;
extern string Pairs = "EURAUD,EURCAD,EURCHF,EURGBP,EURNZD,EURUSD,EURJPY,AUDCAD,AUDCHF,AUDJPY,AUDNZD,AUDUSD,GBPAUD,GBPCAD,GBPCHF,GBPJPY,GBPNZD,GBPUSD,CADCHF,CADJPY,USDCAD,USDCHF,USDJPY,NZDCAD,NZDCHF,NZDJPY,NZDUSD,CHFJPY";
extern string delimiter = ",";
//+------------------------------------------------------------------+
//| Local Parameters Definition |
//+------------------------------------------------------------------+
datetime lastExport[];
string pairs[];
//+------------------------------------------------------------------+
//| Custom indicator initialization function |
//+------------------------------------------------------------------+
int init()
{
//------------------------------------------------------------------
Split(Pairs, pairs, ",");
//------------------------------------------------------------------
if (ArraySize(pairs) == 0 || StringTrimLeft(StringTrimRight(pairs[0])) == "")
{
Alert("Pairs are not entered correctly please check it...");
return (0);
}
//------------------------------------------------------------------
ArrayResize(lastExport, ArraySize(pairs));
ArrayInitialize(lastExport, 0);
//------------------------------------------------------------------
Comment("quote exporter is active :)");
//------------------------------------------------------------------
return(0);
}
//+------------------------------------------------------------------+
//| Custom indicator deinitialization function |
//+------------------------------------------------------------------+
int deinit()
{
//------------------------------------------------------------------
Comment("");
//------------------------------------------------------------------
return(0);
}
//+------------------------------------------------------------------+
//| Custom indicator iteration function |
//+------------------------------------------------------------------+
int start()
{
//------------------------------------------------------------------
if (ArraySize(pairs) == 0 || StringTrimLeft(StringTrimRight(pairs[0])) == "") return (0);
//------------------------------------------------------------------
BarCount = MathMin(Bars, BarCount);
//------------------------------------------------------------------
for (int j = 0; j < ArraySize(pairs); j++)
{
if (lastExport[j] == Time[0]) continue;
lastExport[j] = Time[0];
if (StringTrimLeft(StringTrimRight(pairs[j])) == "") continue;
if (MarketInfo(pairs[j], MODE_BID) == 0) { Alert("symbol " + pairs[j] + " is not loaded!!!"); continue; }
//------------------------------------------------------------------
string file = pairs[j] + "_" + GetTimeFrameName(0) + ".csv";
int log = FileOpen(file, FILE_CSV|FILE_WRITE, "~");
if (log < 0) { Alert("can not create/overwrite csv file " + file + "!!!"); continue; }
string buffer;
buffer = "Date"+delimiter+"Time"+delimiter+"Open"+delimiter+"High"+delimiter+"Low"+delimiter+"Close"+delimiter+"Volume";
FileWrite(log, buffer);
int digits = MarketInfo(pairs[j], MODE_DIGITS);
for (int i = BarCount; i >= 1; i--)
{
buffer = TimeToStr(Time[i], TIME_DATE)+delimiter+TimeToStr(Time[i], TIME_MINUTES)+delimiter+DoubleToStr(iOpen(pairs[j], 0, i), digits)+delimiter+DoubleToStr(iHigh(pairs[j], 0, i), digits)+delimiter+DoubleToStr(iLow(pairs[j], 0, i), digits)+delimiter+DoubleToStr(iClose(pairs[j], 0, i), digits)+delimiter+DoubleToStr(iVolume(pairs[j], 0, i), 0);
FileWrite(log, buffer);
}
buffer = "0"+delimiter+"0"+delimiter+"0"+delimiter+"0"+delimiter+"0"+delimiter+"0"+delimiter+"0";
FileWrite(log, buffer);
FileClose(log);
}
//------------------------------------------------------------------
return(0);
}
//+------------------------------------------------------------------+
string GetTimeFrameName(int TimeFrame)
{
switch (TimeFrame)
{
case PERIOD_M1: return("M1");
case PERIOD_M5: return("M5");
case PERIOD_M15: return("M15");
case PERIOD_M30: return("M30");
case PERIOD_H1: return("H1");
case PERIOD_H4: return("H4");
case PERIOD_D1: return("D1");
case PERIOD_W1: return("W1");
case PERIOD_MN1: return("MN1");
case 0: return(GetTimeFrameName(Period()));
}
}
//+------------------------------------------------------------------+
void Split(string buffer, string &splitted[], string separator)
{
string value = "";
int index = 0;
ArrayResize(splitted, 0);
if (StringSubstr(buffer, StringLen(buffer) - 1) != separator) buffer = buffer + separator;
for (int i = 0; i < StringLen(buffer); i++)
if (StringSubstr(buffer, i, 1) == separator)
{
ArrayResize(splitted, index + 1);
splitted[index] = value;
index ++;
value = "";
}
else
value = value + StringSubstr(buffer, i, 1);
}
//+------------------------------------------------------------------+

MQL5: how do I automatically delete all un-triggered pending orders before placing new orders?

I am working on a Project that requires me to place a BUYSTOP and a SELLSTOP pair of orders and then on the next bar if those orders are not triggered, then delete them and place fresh ones.
Here is my code:
if(logic == true && OrdersTotal() == 0)
{bool res = OrderSend(....);}
if(OrdersTotal() != 0)
{
if(ordertype == OP_BUY || ordertype == OP_SELL)
{
bool del = OrderDelete(....);
}
}
This code is properly placing orders and deleting them as well when I am testing.
But when the EA is active on the live server, it does not open orders because the platform already has orders of other instruments open.
I'm sure there would be quite an easy way to get around this but since I am a novice, I'm not able to figure that out.
it is not clear whether you use magic number and symbol check.
you should check sth like
int _ordersTotal = OrdersTotal()-1;
for (int i = _ordersTotal; i >= 0; i--){
if (OrderSymbol() != Symbol() || OrderMagicNumber() != magic) continue;
....
}
in different realization, i.e. you can create a function(String Symbol) that checks if you have some working orders placed of the indicated Symbol.
Old proverb says:Measure twice before cut onceThere are actually four values ( three for pendings ) to check before OrderDelete()
As your definition states to handle { OP_{BUY|SELL}STOP } orders, there are following three items to check:
Symbol() match ( not causing unwanted side-effects by deleting other EA's or manual orders )
OrderType() match ( not ignoring the actual state { PENDING | AT_MARKET } and direction { BUY | SELL } of the order )
OrderMagicNumber() match ( not ignoring the UUID selector utility available to be set for each individual OrderSend() )
So, let's sketch the detection process:
int myEaContextAwareMagicNUMBER = ...;
for ( int ii = OrdersTotal();
ii >= 0;
ii--
)
if OrderSelect( ii, SELECT_BY_POS, MODE_TRADES )
{
if ( OrderSymbol() != _Symbol
&& OrderMagicNumber() != myEaContextAwareMagicNUMBER
&& OrderOpenTime() >= Time[1] // Prev. Bar
&& !( OrderType() == OP_BUYSTOP
|| OrderType() == OP_SELLSTOP
)
) continue; // __^ __^ __^ __^ __^ __^ loop for next test
// -------------------------------------------+
// FINALLY PROCESS THE MATCHING OrderDelete() |
// -------------------------------------------+
...
..
.
// -------------------------------------------+
}
else Print( "WARN: OrderSelect() failed at db.POOL.SELECT(), RECORD_NUMBER == ", ii );
So how to delete un-triggered pendings is done.
Next comes the remark about
"... when the ea is active on the live server, it does not open orders because the platform already has orders of other instruments open."
There could hardly be any advice provided without delivering the exact { GetLastError() | _LastError } values.
Some Brokers for some account types do indeed restrict OrderSend() acceptance policies, and thus besides the GetLastError() value the respective Broker Terms and Conditions apply.
Do not hesitate to ask more & may enjoy other Questions/Answers in MQL4 domain.

Jump to listbox item by typing first few characters

I have a list of items (potentially large) from which the user must select one. I'd like to allow the user to type the first few letters of the desired item to jump to the correct place in the list. By default, each keypress jumps to the first item starting with that letter, so you can't type the first several letters. Is there any straightforward way to do this? Any CodeProject or other such example?
I've looked for hours, and found any number of samples for IAutocomplete, but that won't help here because I need to guarantee that the result is in the list.
The only way I can think to do this is to derive from CListBox, capture the keystrokes myself, find the item, run a timer so that new keystrokes after a sufficient pause will start a new search... since I'm not an MFC jock, this is daunting. Any tips much appreciated.
One clarifying note: my ultimate goal is actually to get this keyboard behavior for a ComboBox of DropDownList style (i.e. no edit box). The lack of an edit box rules out most autocomplete code, and the need for ComboBox functionality means I can't use CListCtrl by itself.
After much unnecessary pain, I've discovered that the real correct answer is simply to use LBS_SORT. Simply by specifying this style, the basic vanilla listbox supports the incremental search keyboard shortcut style I wanted. Without LBS_SORT (or CBS_SORT for a combobox), you get the irritating and almost-useless jump-to-first-letter-only behavior. I didn't try LBS_SORT because my list contents were added in sorted order anyway.
So the dozen or so hours of investigating custom controls, etc., all for naught because the Microsoft documentation makes no mention of this important behavioral difference in the description of LBS_SORT!!
Thanks to everyone who contributed.
I've implemented such a functionality in core Win32. Heres the code.
Somewhere in your message loop that processes the list box insert:
switch(message)
{
case WM_CHAR:
if(HandleListBoxKeyStrokes(hwnd, wParam) == FALSE)
return FALSE;
....
Heres the code (propably not fully complete):
/* ======================================================================== */
/* ======================================================================== */
#define RETURNr(a, b) // homegrown asserts
BOOLEAN HandleListBoxKeyStrokes(HWND hwnd, UINT theKey)
{
#define MAXCHARCACHEINTERVALL 600.0 // Max. milisecs time offset to consider as typed 'at once'
static char sgLastChars[255] = {'0'};
static double sgLastCharTime = 0.;
static HWND sgLasthwnd = NULL;
if(GetSecs() - sgLastCharTime > MAXCHARCACHEINTERVALL ||
sgLasthwnd != hwnd)
*sgLastChars = 0;
if(theKey == ' ' && *sgLastChars == 0)
return TRUE;
sgLastCharTime = GetSecs();
sgLasthwnd = hwnd;
AppendChar(sgLastChars, toupper(theKey));
if(strlen(sgLastChars) > 1)
{
LONG l = GetWindowLong(hwnd, GWL_STYLE);
Char255 tx;
GetClassName(hwnd, tx, sizeof(tx));
if( (! stricmp(tx, "Listbox") &&
! (l & (LBS_EXTENDEDSEL | LBS_MULTIPLESEL)) ) ||
(! stricmp(tx, "ComboBox") && // combo Box support
l & CBS_DROPDOWNLIST &&
! (l & (CBS_OWNERDRAWFIXED | CBS_OWNERDRAWVARIABLE)) ) )
{
long Count, l, BestMatch = - 1, BestMatchOff = 0;
long LBcmdSet[] = {LB_GETCOUNT, LB_GETTEXTLEN , LB_GETTEXT};
long CBcmdSet[] = {CB_GETCOUNT, CB_GETLBTEXTLEN, CB_GETLBTEXT};
long *cmdSet = (! stricmp(tx, "ComboBox")) ? CBcmdSet : LBcmdSet;
RETURNr((Count = SendMessage(hwnd, cmdSet[0], 0, 0)) != LB_ERR, 0);
for(int i = 0; i < Count; i++)
{
RETURNr((l = SendMessage(hwnd, cmdSet[1], i, 0)) != LB_ERR, TRUE);
RETURNr( l < sizeof(tx), TRUE);
RETURNr((l = SendMessage(hwnd, cmdSet[2], i, (LPARAM)&tx)) != LB_ERR, TRUE);
strupr(tx);
if(! strncmp(tx, sgLastChars, strlen(sgLastChars)))
{
SelListBoxAndNotify(hwnd, i);
return FALSE;
}
char *p;
if(p = strstr(tx, sgLastChars))
{
int off = p - tx;
if(BestMatch == -1 || off < BestMatchOff)
{
BestMatch = i;
BestMatchOff = off;
}
}
}
// If text not found at start of string see if it matches some part inside the string
if(BestMatch != -1)
SelListBoxAndNotify(hwnd, BestMatch);
// Nothing found - dont process
return FALSE;
}
}
return TRUE;
}
/* ======================================================================== */
/* ======================================================================== */
void SelListBoxAndNotify(HWND hwnd, int index)
{
// i am sorry here - this is some XVT-toolkit specific code.
// it has to be replaced with something similar for native Win32
WINDOW win = xvtwi_hwnd_to_window(hwnd);
WINDOW parent = xvt_vobj_get_parent(win);
xvt_list_set_sel(win, index, 1);
EVENT evt;
memset(&evt, 0, sizeof(evt));
evt.type = E_CONTROL;
evt.v.ctl.id = GetDlgCtrlID(hwnd);
evt.v.ctl.ci.v.lbox.dbl_click = FALSE;
xvt_win_dispatch_event(parent, &evt);
}
/* ======================================================================== */
/* ======================================================================== */
double GetSecs(void)
{
struct timeb timebuffer;
ftime(&timebuffer);
return (double)timebuffer.millitm +
((double)timebuffer.time * 1000.) - // Timezone needed for DbfGetToday
((double)timebuffer.timezone * 60. * 1000.);
}
/* ======================================================================== */
/* ======================================================================== */
char AppendChar(char *tx, char C)
{ int i;
i = strlen(tx);
tx[i ] = C;
tx[i + 1] = 0;
return(C);
}
Can you use a CListView CListCtrl instead? They work like that by default.

How do I create a URL shortener? [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Closed 1 year ago.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
I want to create a URL shortener service where you can write a long URL into an input field and the service shortens the URL to "http://www.example.org/abcdef".
Instead of "abcdef" there can be any other string with six characters containing a-z, A-Z and 0-9. That makes 56~57 billion possible strings.
My approach:
I have a database table with three columns:
id, integer, auto-increment
long, string, the long URL the user entered
short, string, the shortened URL (or just the six characters)
I would then insert the long URL into the table. Then I would select the auto-increment value for "id" and build a hash of it. This hash should then be inserted as "short". But what sort of hash should I build? Hash algorithms like MD5 create too long strings. I don't use these algorithms, I think. A self-built algorithm will work, too.
My idea:
For "http://www.google.de/" I get the auto-increment id 239472. Then I do the following steps:
short = '';
if divisible by 2, add "a"+the result to short
if divisible by 3, add "b"+the result to short
... until I have divisors for a-z and A-Z.
That could be repeated until the number isn't divisible any more. Do you think this is a good approach? Do you have a better idea?
Due to the ongoing interest in this topic, I've published an efficient solution to GitHub, with implementations for JavaScript, PHP, Python and Java. Add your solutions if you like :)
I would continue your "convert number to string" approach. However, you will realize that your proposed algorithm fails if your ID is a prime and greater than 52.
Theoretical background
You need a Bijective Function f. This is necessary so that you can find a inverse function g('abc') = 123 for your f(123) = 'abc' function. This means:
There must be no x1, x2 (with x1 ≠ x2) that will make f(x1) = f(x2),
and for every y you must be able to find an x so that f(x) = y.
How to convert the ID to a shortened URL
Think of an alphabet we want to use. In your case, that's [a-zA-Z0-9]. It contains 62 letters.
Take an auto-generated, unique numerical key (the auto-incremented id of a MySQL table for example).
For this example, I will use 12510 (125 with a base of 10).
Now you have to convert 12510 to X62 (base 62).
12510 = 2×621 + 1×620 = [2,1]
This requires the use of integer division and modulo. A pseudo-code example:
digits = []
while num > 0
remainder = modulo(num, 62)
digits.push(remainder)
num = divide(num, 62)
digits = digits.reverse
Now map the indices 2 and 1 to your alphabet. This is how your mapping (with an array for example) could look like:
0 → a
1 → b
...
25 → z
...
52 → 0
61 → 9
With 2 → c and 1 → b, you will receive cb62 as the shortened URL.
http://shor.ty/cb
How to resolve a shortened URL to the initial ID
The reverse is even easier. You just do a reverse lookup in your alphabet.
e9a62 will be resolved to "4th, 61st, and 0th letter in the alphabet".
e9a62 = [4,61,0] = 4×622 + 61×621 + 0×620 = 1915810
Now find your database-record with WHERE id = 19158 and do the redirect.
Example implementations (provided by commenters)
C++
Python
Ruby
Haskell
C#
CoffeeScript
Perl
Why would you want to use a hash?
You can just use a simple translation of your auto-increment value to an alphanumeric value. You can do that easily by using some base conversion. Say you character space (A-Z, a-z, 0-9, etc.) has 62 characters, convert the id to a base-40 number and use the characters as the digits.
public class UrlShortener {
private static final String ALPHABET = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
private static final int BASE = ALPHABET.length();
public static String encode(int num) {
StringBuilder sb = new StringBuilder();
while ( num > 0 ) {
sb.append( ALPHABET.charAt( num % BASE ) );
num /= BASE;
}
return sb.reverse().toString();
}
public static int decode(String str) {
int num = 0;
for ( int i = 0; i < str.length(); i++ )
num = num * BASE + ALPHABET.indexOf(str.charAt(i));
return num;
}
}
Not an answer to your question, but I wouldn't use case-sensitive shortened URLs. They are hard to remember, usually unreadable (many fonts render 1 and l, 0 and O and other characters very very similar that they are near impossible to tell the difference) and downright error prone. Try to use lower or upper case only.
Also, try to have a format where you mix the numbers and characters in a predefined form. There are studies that show that people tend to remember one form better than others (think phone numbers, where the numbers are grouped in a specific form). Try something like num-char-char-num-char-char. I know this will lower the combinations, especially if you don't have upper and lower case, but it would be more usable and therefore useful.
My approach: Take the Database ID, then Base36 Encode it. I would NOT use both Upper AND Lowercase letters, because that makes transmitting those URLs over the telephone a nightmare, but you could of course easily extend the function to be a base 62 en/decoder.
Here is my PHP 5 class.
<?php
class Bijective
{
public $dictionary = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
public function __construct()
{
$this->dictionary = str_split($this->dictionary);
}
public function encode($i)
{
if ($i == 0)
return $this->dictionary[0];
$result = '';
$base = count($this->dictionary);
while ($i > 0)
{
$result[] = $this->dictionary[($i % $base)];
$i = floor($i / $base);
}
$result = array_reverse($result);
return join("", $result);
}
public function decode($input)
{
$i = 0;
$base = count($this->dictionary);
$input = str_split($input);
foreach($input as $char)
{
$pos = array_search($char, $this->dictionary);
$i = $i * $base + $pos;
}
return $i;
}
}
A Node.js and MongoDB solution
Since we know the format that MongoDB uses to create a new ObjectId with 12 bytes.
a 4-byte value representing the seconds since the Unix epoch,
a 3-byte machine identifier,
a 2-byte process id
a 3-byte counter (in your machine), starting with a random value.
Example (I choose a random sequence)
a1b2c3d4e5f6g7h8i9j1k2l3
a1b2c3d4 represents the seconds since the Unix epoch,
4e5f6g7 represents machine identifier,
h8i9 represents process id
j1k2l3 represents the counter, starting with a random value.
Since the counter will be unique if we are storing the data in the same machine we can get it with no doubts that it will be duplicate.
So the short URL will be the counter and here is a code snippet assuming that your server is running properly.
const mongoose = require('mongoose');
const Schema = mongoose.Schema;
// Create a schema
const shortUrl = new Schema({
long_url: { type: String, required: true },
short_url: { type: String, required: true, unique: true },
});
const ShortUrl = mongoose.model('ShortUrl', shortUrl);
// The user can request to get a short URL by providing a long URL using a form
app.post('/shorten', function(req ,res){
// Create a new shortUrl */
// The submit form has an input with longURL as its name attribute.
const longUrl = req.body["longURL"];
const newUrl = ShortUrl({
long_url : longUrl,
short_url : "",
});
const shortUrl = newUrl._id.toString().slice(-6);
newUrl.short_url = shortUrl;
console.log(newUrl);
newUrl.save(function(err){
console.log("the new URL is added");
})
});
I keep incrementing an integer sequence per domain in the database and use Hashids to encode the integer into a URL path.
static hashids = Hashids(salt = "my app rocks", minSize = 6)
I ran a script to see how long it takes until it exhausts the character length. For six characters it can do 164,916,224 links and then goes up to seven characters. Bitly uses seven characters. Under five characters looks weird to me.
Hashids can decode the URL path back to a integer but a simpler solution is to use the entire short link sho.rt/ka8ds3 as a primary key.
Here is the full concept:
function addDomain(domain) {
table("domains").insert("domain", domain, "seq", 0)
}
function addURL(domain, longURL) {
seq = table("domains").where("domain = ?", domain).increment("seq")
shortURL = domain + "/" + hashids.encode(seq)
table("links").insert("short", shortURL, "long", longURL)
return shortURL
}
// GET /:hashcode
function handleRequest(req, res) {
shortURL = req.host + "/" + req.param("hashcode")
longURL = table("links").where("short = ?", shortURL).get("long")
res.redirect(301, longURL)
}
C# version:
public class UrlShortener
{
private static String ALPHABET = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
private static int BASE = 62;
public static String encode(int num)
{
StringBuilder sb = new StringBuilder();
while ( num > 0 )
{
sb.Append( ALPHABET[( num % BASE )] );
num /= BASE;
}
StringBuilder builder = new StringBuilder();
for (int i = sb.Length - 1; i >= 0; i--)
{
builder.Append(sb[i]);
}
return builder.ToString();
}
public static int decode(String str)
{
int num = 0;
for ( int i = 0, len = str.Length; i < len; i++ )
{
num = num * BASE + ALPHABET.IndexOf( str[(i)] );
}
return num;
}
}
You could hash the entire URL, but if you just want to shorten the id, do as marcel suggested. I wrote this Python implementation:
https://gist.github.com/778542
Take a look at https://hashids.org/ it is open source and in many languages.
Their page outlines some of the pitfalls of other approaches.
If you don't want re-invent the wheel ... http://lilurl.sourceforge.net/
// simple approach
$original_id = 56789;
$shortened_id = base_convert($original_id, 10, 36);
$un_shortened_id = base_convert($shortened_id, 36, 10);
alphabet = map(chr, range(97,123)+range(65,91)) + map(str,range(0,10))
def lookup(k, a=alphabet):
if type(k) == int:
return a[k]
elif type(k) == str:
return a.index(k)
def encode(i, a=alphabet):
'''Takes an integer and returns it in the given base with mappings for upper/lower case letters and numbers 0-9.'''
try:
i = int(i)
except Exception:
raise TypeError("Input must be an integer.")
def incode(i=i, p=1, a=a):
# Here to protect p.
if i <= 61:
return lookup(i)
else:
pval = pow(62,p)
nval = i/pval
remainder = i % pval
if nval <= 61:
return lookup(nval) + incode(i % pval)
else:
return incode(i, p+1)
return incode()
def decode(s, a=alphabet):
'''Takes a base 62 string in our alphabet and returns it in base10.'''
try:
s = str(s)
except Exception:
raise TypeError("Input must be a string.")
return sum([lookup(i) * pow(62,p) for p,i in enumerate(list(reversed(s)))])a
Here's my version for whomever needs it.
Why not just translate your id to a string? You just need a function that maps a digit between, say, 0 and 61 to a single letter (upper/lower case) or digit. Then apply this to create, say, 4-letter codes, and you've got 14.7 million URLs covered.
Here is a decent URL encoding function for PHP...
// From http://snipplr.com/view/22246/base62-encode--decode/
private function base_encode($val, $base=62, $chars='0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ') {
$str = '';
do {
$i = fmod($val, $base);
$str = $chars[$i] . $str;
$val = ($val - $i) / $base;
} while($val > 0);
return $str;
}
Don't know if anyone will find this useful - it is more of a 'hack n slash' method, yet is simple and works nicely if you want only specific chars.
$dictionary = "abcdfghjklmnpqrstvwxyz23456789";
$dictionary = str_split($dictionary);
// Encode
$str_id = '';
$base = count($dictionary);
while($id > 0) {
$rem = $id % $base;
$id = ($id - $rem) / $base;
$str_id .= $dictionary[$rem];
}
// Decode
$id_ar = str_split($str_id);
$id = 0;
for($i = count($id_ar); $i > 0; $i--) {
$id += array_search($id_ar[$i-1], $dictionary) * pow($base, $i - 1);
}
Did you omit O, 0, and i on purpose?
I just created a PHP class based on Ryan's solution.
<?php
$shorty = new App_Shorty();
echo 'ID: ' . 1000;
echo '<br/> Short link: ' . $shorty->encode(1000);
echo '<br/> Decoded Short Link: ' . $shorty->decode($shorty->encode(1000));
/**
* A nice shorting class based on Ryan Charmley's suggestion see the link on Stack Overflow below.
* #author Svetoslav Marinov (Slavi) | http://WebWeb.ca
* #see http://stackoverflow.com/questions/742013/how-to-code-a-url-shortener/10386945#10386945
*/
class App_Shorty {
/**
* Explicitly omitted: i, o, 1, 0 because they are confusing. Also use only lowercase ... as
* dictating this over the phone might be tough.
* #var string
*/
private $dictionary = "abcdfghjklmnpqrstvwxyz23456789";
private $dictionary_array = array();
public function __construct() {
$this->dictionary_array = str_split($this->dictionary);
}
/**
* Gets ID and converts it into a string.
* #param int $id
*/
public function encode($id) {
$str_id = '';
$base = count($this->dictionary_array);
while ($id > 0) {
$rem = $id % $base;
$id = ($id - $rem) / $base;
$str_id .= $this->dictionary_array[$rem];
}
return $str_id;
}
/**
* Converts /abc into an integer ID
* #param string
* #return int $id
*/
public function decode($str_id) {
$id = 0;
$id_ar = str_split($str_id);
$base = count($this->dictionary_array);
for ($i = count($id_ar); $i > 0; $i--) {
$id += array_search($id_ar[$i - 1], $this->dictionary_array) * pow($base, $i - 1);
}
return $id;
}
}
?>
public class TinyUrl {
private final String characterMap = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
private final int charBase = characterMap.length();
public String covertToCharacter(int num){
StringBuilder sb = new StringBuilder();
while (num > 0){
sb.append(characterMap.charAt(num % charBase));
num /= charBase;
}
return sb.reverse().toString();
}
public int covertToInteger(String str){
int num = 0;
for(int i = 0 ; i< str.length(); i++)
num += characterMap.indexOf(str.charAt(i)) * Math.pow(charBase , (str.length() - (i + 1)));
return num;
}
}
class TinyUrlTest{
public static void main(String[] args) {
TinyUrl tinyUrl = new TinyUrl();
int num = 122312215;
String url = tinyUrl.covertToCharacter(num);
System.out.println("Tiny url: " + url);
System.out.println("Id: " + tinyUrl.covertToInteger(url));
}
}
This is what I use:
# Generate a [0-9a-zA-Z] string
ALPHABET = map(str,range(0, 10)) + map(chr, range(97, 123) + range(65, 91))
def encode_id(id_number, alphabet=ALPHABET):
"""Convert an integer to a string."""
if id_number == 0:
return alphabet[0]
alphabet_len = len(alphabet) # Cache
result = ''
while id_number > 0:
id_number, mod = divmod(id_number, alphabet_len)
result = alphabet[mod] + result
return result
def decode_id(id_string, alphabet=ALPHABET):
"""Convert a string to an integer."""
alphabet_len = len(alphabet) # Cache
return sum([alphabet.index(char) * pow(alphabet_len, power) for power, char in enumerate(reversed(id_string))])
It's very fast and can take long integers.
For a similar project, to get a new key, I make a wrapper function around a random string generator that calls the generator until I get a string that hasn't already been used in my hashtable. This method will slow down once your name space starts to get full, but as you have said, even with only 6 characters, you have plenty of namespace to work with.
I have a variant of the problem, in that I store web pages from many different authors and need to prevent discovery of pages by guesswork. So my short URLs add a couple of extra digits to the Base-62 string for the page number. These extra digits are generated from information in the page record itself and they ensure that only 1 in 3844 URLs are valid (assuming 2-digit Base-62). You can see an outline description at http://mgscan.com/MBWL.
Very good answer, I have created a Golang implementation of the bjf:
package bjf
import (
"math"
"strings"
"strconv"
)
const alphabet = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
func Encode(num string) string {
n, _ := strconv.ParseUint(num, 10, 64)
t := make([]byte, 0)
/* Special case */
if n == 0 {
return string(alphabet[0])
}
/* Map */
for n > 0 {
r := n % uint64(len(alphabet))
t = append(t, alphabet[r])
n = n / uint64(len(alphabet))
}
/* Reverse */
for i, j := 0, len(t) - 1; i < j; i, j = i + 1, j - 1 {
t[i], t[j] = t[j], t[i]
}
return string(t)
}
func Decode(token string) int {
r := int(0)
p := float64(len(token)) - 1
for i := 0; i < len(token); i++ {
r += strings.Index(alphabet, string(token[i])) * int(math.Pow(float64(len(alphabet)), p))
p--
}
return r
}
Hosted at github: https://github.com/xor-gate/go-bjf
Implementation in Scala:
class Encoder(alphabet: String) extends (Long => String) {
val Base = alphabet.size
override def apply(number: Long) = {
def encode(current: Long): List[Int] = {
if (current == 0) Nil
else (current % Base).toInt :: encode(current / Base)
}
encode(number).reverse
.map(current => alphabet.charAt(current)).mkString
}
}
class Decoder(alphabet: String) extends (String => Long) {
val Base = alphabet.size
override def apply(string: String) = {
def decode(current: Long, encodedPart: String): Long = {
if (encodedPart.size == 0) current
else decode(current * Base + alphabet.indexOf(encodedPart.head),encodedPart.tail)
}
decode(0,string)
}
}
Test example with Scala test:
import org.scalatest.{FlatSpec, Matchers}
class DecoderAndEncoderTest extends FlatSpec with Matchers {
val Alphabet = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
"A number with base 10" should "be correctly encoded into base 62 string" in {
val encoder = new Encoder(Alphabet)
encoder(127) should be ("cd")
encoder(543513414) should be ("KWGPy")
}
"A base 62 string" should "be correctly decoded into a number with base 10" in {
val decoder = new Decoder(Alphabet)
decoder("cd") should be (127)
decoder("KWGPy") should be (543513414)
}
}
Function based in Xeoncross Class
function shortly($input){
$dictionary = ['a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z','A','B','C','D','E','F','G','H','I','J','K','L','M','N','O','P','Q','R','S','T','U','V','W','X','Y','Z','0','1','2','3','4','5','6','7','8','9'];
if($input===0)
return $dictionary[0];
$base = count($dictionary);
if(is_numeric($input)){
$result = [];
while($input > 0){
$result[] = $dictionary[($input % $base)];
$input = floor($input / $base);
}
return join("", array_reverse($result));
}
$i = 0;
$input = str_split($input);
foreach($input as $char){
$pos = array_search($char, $dictionary);
$i = $i * $base + $pos;
}
return $i;
}
Here is a Node.js implementation that is likely to bit.ly. generate a highly random seven-character string.
It uses Node.js crypto to generate a highly random 25 charset rather than randomly selecting seven characters.
var crypto = require("crypto");
exports.shortURL = new function () {
this.getShortURL = function () {
var sURL = '',
_rand = crypto.randomBytes(25).toString('hex'),
_base = _rand.length;
for (var i = 0; i < 7; i++)
sURL += _rand.charAt(Math.floor(Math.random() * _rand.length));
return sURL;
};
}
My Python 3 version
base_list = list("0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ")
base = len(base_list)
def encode(num: int):
result = []
if num == 0:
result.append(base_list[0])
while num > 0:
result.append(base_list[num % base])
num //= base
print("".join(reversed(result)))
def decode(code: str):
num = 0
code_list = list(code)
for index, code in enumerate(reversed(code_list)):
num += base_list.index(code) * base ** index
print(num)
if __name__ == '__main__':
encode(341413134141)
decode("60FoItT")
For a quality Node.js / JavaScript solution, see the id-shortener module, which is thoroughly tested and has been used in production for months.
It provides an efficient id / URL shortener backed by pluggable storage defaulting to Redis, and you can even customize your short id character set and whether or not shortening is idempotent. This is an important distinction that not all URL shorteners take into account.
In relation to other answers here, this module implements the Marcel Jackwerth's excellent accepted answer above.
The core of the solution is provided by the following Redis Lua snippet:
local sequence = redis.call('incr', KEYS[1])
local chars = '0123456789ABCDEFGHJKLMNPQRSTUVWXYZ_abcdefghijkmnopqrstuvwxyz'
local remaining = sequence
local slug = ''
while (remaining > 0) do
local d = (remaining % 60)
local character = string.sub(chars, d + 1, d + 1)
slug = character .. slug
remaining = (remaining - d) / 60
end
redis.call('hset', KEYS[2], slug, ARGV[1])
return slug
Why not just generate a random string and append it to the base URL? This is a very simplified version of doing this in C#.
static string chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890";
static string baseUrl = "https://google.com/";
private static string RandomString(int length)
{
char[] s = new char[length];
Random rnd = new Random();
for (int x = 0; x < length; x++)
{
s[x] = chars[rnd.Next(chars.Length)];
}
Thread.Sleep(10);
return new String(s);
}
Then just add the append the random string to the baseURL:
string tinyURL = baseUrl + RandomString(5);
Remember this is a very simplified version of doing this and it's possible the RandomString method could create duplicate strings. In production you would want to take in account for duplicate strings to ensure you will always have a unique URL. I have some code that takes account for duplicate strings by querying a database table I could share if anyone is interested.
This is my initial thoughts, and more thinking can be done, or some simulation can be made to see if it works well or any improvement is needed:
My answer is to remember the long URL in the database, and use the ID 0 to 9999999999999999 (or however large the number is needed).
But the ID 0 to 9999999999999999 can be an issue, because
it can be shorter if we use hexadecimal, or even base62 or base64. (base64 just like YouTube using A-Z a-z 0-9 _ and -)
if it increases from 0 to 9999999999999999 uniformly, then hackers can visit them in that order and know what URLs people are sending each other, so it can be a privacy issue
We can do this:
have one server allocate 0 to 999 to one server, Server A, so now Server A has 1000 of such IDs. So if there are 20 or 200 servers constantly wanting new IDs, it doesn't have to keep asking for each new ID, but rather asking once for 1000 IDs
for the ID 1, for example, reverse the bits. So 000...00000001 becomes 10000...000, so that when converted to base64, it will be non-uniformly increasing IDs each time.
use XOR to flip the bits for the final IDs. For example, XOR with 0xD5AA96...2373 (like a secret key), and the some bits will be flipped. (whenever the secret key has the 1 bit on, it will flip the bit of the ID). This will make the IDs even harder to guess and appear more random
Following this scheme, the single server that allocates the IDs can form the IDs, and so can the 20 or 200 servers requesting the allocation of IDs. The allocating server has to use a lock / semaphore to prevent two requesting servers from getting the same batch (or if it is accepting one connection at a time, this already solves the problem). So we don't want the line (queue) to be too long for waiting to get an allocation. So that's why allocating 1000 or 10000 at a time can solve the issue.

Resources