% Upper-case    A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
% Lower-case    a b c d e f g h i j k l m n o p q r s t u v w x y z
% Digits        0 1 2 3 4 5 6 7 8 9
% Exclamation   !           Double quote "          Hash (number) #
% Dollar        $           Percent      %          Ampersand     &
% Acute accent  '           Left paren   (          Right paren   )
% Asterisk      *           Plus         +          Comma         ,
% Minus         -           Point        .          Solidus       /
% Colon         :           Semicolon    ;          Less than     <
% Equals        =           Greater than >          Question mark ?
% At            @           Left bracket [          Backslash     \
% Right bracket ]           Circumflex   ^          Underscore    _
% Grave accent  `           Left brace   {          Vertical bar  |
% Right brace   }           Tilde        ~

% ---------------------------------------------------------------------|
% --------------------------- 72 characters ---------------------------|
% ---------------------------------------------------------------------|
%
% Optimal Foraging Theory Revisited: (old) Chapter 3. Review of
%                                    Classical Optimal Foraging Theory
% (this material not in final version of document; papers based on this
%  material may be generated later)
%
% (c) Copyright 2007 by Theodore P. Pavlic
%

\chapter{Review of Classical Optimal Foraging Theory}
\label{ch:review_classical_oft}

In this \chname{}, we review the classical \ac{OFT} approach to
optimization. Where possible, we will use notation from
\longref{ch:model}; this will let us examine strengths and weaknesses of
the classical \ac{OFT} approach and give a basis for comparison to our
approach. In \longref{sec:oft_long_term_rate}, we present the
optimization criterion used by classical \ac{OFT} and the justifications
for it.
%We present and respond to the conventional criticisms of
%classical \ac{OFT} in \longref{sec:oft_criticisms}.

\section{OFT Focus: Long-Term Rate of Net Gain}
\label{sec:oft_long_term_rate}

In biological contexts, it is expected that natural selection will favor
foraging behaviors that provide greater future reproductive success, a
common surrogate for Darwinian fitness. So, functions mapping specific
behaviors to quantitative measures of reproductive success can be
optimized to predict behaviors should result from a long run of natural
selection. \Citet{Schoener71} defines such a model, and while quantities
in the model are too difficult to define for most cases, behaviors
predicted by the model fall on a continuum from foraging time minimizers
(when energy is held constant) to energy maximizers (when foraging time
is held constant). \Citet{PPC77} argue that the \emph{rate} of net
energy intake is the most general function to be maximized as it
captures both extremes on the \citeauthor{Schoener71} continuum by
asserting an upward pressure on energy intake and a downward pressure on
foraging time (assuming net energetic intake is always positive). This
will allow a forager to achieve its energy consumption needs while also
leaving it enough time for other activities such as reproduction and
predator avoidance. Defining the rate of net energy intake can be done
in a number of different ways. Using the terms from \longref{ch:model},
it could be defined for any $t \in \R_{\geq0}$ or $N \in \N$ as
$\oft{G}^N/\oft{T}^N$ or $\E( \oft{G}^N/\oft{T}^N )$ or $\oft{G}(t)/t$
or $\E( \oft{G}(t) )/t$. However, \citeauthor{PPC77} also argue that
rates should be calculated over the entire life history of the forager.
Thus, rather than taking a particular $t \in \R_{\geq0}$ or $N \in \N$,
the asymptotic limits can be taken. Conveniently,
\longrefs{eq:oft_payoff_long_rate} shows that all of these are
equivalent. In fact, by \longref{eq:oft_payoff_long_rate_equiv}, 
%
\begin{equation} 
        \begin{split}
        \frac{\E(\oft{G}_1)}{\E(\oft{T}_1)} 
        &= 
        \frac{ \E(\oft{G}^{N_*}) }{\E(\oft{T}^{N_*})}
        =
        \aslim\limits_{N \to \infty} \frac{\oft{G}^N}{\oft{T}^N} 
        =
        \lim\limits_{N \to \infty} 
        \E\left(\frac{\oft{G}^N}{\oft{T}^N}\right)\\
        &=
        \frac{\E(\oft{G}(t_*))}{\E(\oft{T}(t_*))} = 
        \aslim\limits_{t \to \infty} \frac{\oft{G}(t)}{t} =
        \lim\limits_{t \to \infty} \frac{\E( \oft{G}(t) )}{t}\\
        \end{split}
        \label{eq:oft_payoff_long_rate_summary} 
\end{equation}
%
for any $t_* \in \R_{>0}$ and $N_* \in \N$. For this reason, the ratio
$\E(\oft{G}_1/\E(\oft{T}_1)$ has received significant interest in
classical \ac{OFT} \citep[\eg,][]{HouMc99,SC82,SK86}. We call this ratio
the \emph{long-term (average) rate of net gain}. Note that by
\longref{eq:RoE_equivalence} this ratio plays an identical role in our
analysis approach when we consider the asymptotic case.

\subsubsection{Opportunity Cost Interpretation}

%\Citet[ch.~4]{HouMc99} 
\Citet{HouMc99} provide an interesting interpretation of
$\E(\oft{G}_1)/\E(\oft{T}_1)$. They define constant $\gamma^* \in \R$ to
be the maximum value of $\E(\oft{G}_1)/\E(\oft{T}_1)$ (\ie, the
long-term rate of net gain) over the set of feasible agent behaviors.
They then treat rate $\gamma^*$ as factor converting time spent between
encounters to maximum points possible from that time. Therefore,
$\gamma^*$ converts time into its equivalent \emph{opportunity cost}.
They then show that during a single \ac{OFT} cycle the behavior for that
cycle that maximizes
%
\begin{equation}
        \E(\oft{G}_1 - \gamma^* \oft{T}_1)
        \label{eq:HouMc99_approach}
\end{equation}
%
will be the behavior that achieves the maximum long-term rate of gain
$\gamma^*$. So, maximizing the long-term rate of gain is equivalent to
maximizing the per-cycle \emph{gain} after being discounted by the
opportunity cost of the cycle time. Solving for this behavior can only
be done analytically if $\gamma^*$ is known, and so the method of
\citeauthor{HouMc99} numerically solves for the optimal behavior using
iteration, which could be a weakness of this approach. However, it
demonstrates an important interpretation of
$\E(\oft{G}_1)/\E(\oft{T}_1)$ as the \emph{opportunity cost} of
searching.

\subsubsection{Equilibrium Renewal Process as Attractive Alternative}

\Citet{Cha73} note that it is desirable to derive the \emph{equilibrium
renewal process} rate of net gain. That is, introduce a $T_1 \in
\R_{>0}$ and redefine the process to start after $T_1$ foraging time has
past. Hence, runtime $t$ represents the length of the interval
immediately after time $T_1$. The quantity of interest to
\citeauthor{Cha73} is then $\E(G)/t$, which represents the average rate
of net gain returned to an agent when an agent is in equilibrium with
its environment (\ie, after the decay of any initial transients).
However, they point out that this rate is only known for such a process
if it is additionally assumed that the net gain on each \ac{OFT} cycle
is independent of the total time of each \ac{OFT} cycle (in particular,
the processing time of each cycle). In that case, $\E(G)/t$ can also be
expressed as the ratio $\E(\oft{G}_1)/\E(\oft{T}_1)$. Unfortunately, it
is rare that net gain and processing time time will be independent in a
practical system. Analytical results are not available otherwise.
Because of this, when $\E(\oft{G}_1)/\E(\oft{T}_1)$ is used it is
usually assumed to be a limiting case (\ie, a rate over a long time
rather than a short-term rate after a long time).

\section{Classical OFT as a Fallacy of the Averages}
\label{sec:oft_fallacy_of_averages}

\Citet{TL81} suggest that the use of $\E(\oft{G}_1)/\E(\oft{T}_1)$
throughout classical \ac{OFT} is due to \emph{the fallacy of averages}.
This describes the incorrect assumption that for an arbitrary random
vector $\v{X}$ and (measurable) function $f$,
%
\begin{equation}
        \E( f(\v{X}) ) = f( \E( \v{X} ) )
        \label{eq:fallacy_of_averages}
\end{equation}
%
Behavioral ecologists are frequently criticized for committing this
error \citep[\eg{},][]{WPA88}. In many cases, the criticism or the error
itself arises from mistakenly dropping the limits in
\longref{eq:oft_payoff_long_rate_summary}. Sometimes the limiting
argument is implied by phrases like ``long term,'' but this is not
understood by critics. As explained in \longref{sec:oft_long_term_rate},
the ratio $\E(\oft{G}_1)/\E(\oft{T}_1)$ (\ie, the long-term rate of net
pain) is a reasonable optimization objective for evolutionary analysis.
However, the major responses to \citet{TL81} have not presented this
simple evolutionary argument. In fact, many of these responses only show
a misunderstanding of the true justification for using this criterion
and also neglect to point out the actual serious flaws in the argument
of \citeauthor{TL81}. Unfortunately, these incomplete or incorrect
responses have been the source of a great deal of confusion and mistrust
of classical \ac{OFT} in recent work.

\Citeauthor{TL81} criticize \citet{Cha76} and \citet{Pulliam74}, and
that criticism receives counter-arguments from \citet{GGP82} and
\citet{TGS82}. The claims of \citeauthor{TL81} are based in a
misunderstanding of \citet{Cha76}; unfortunately, while the responses
given by \citeauthor{GGP82} and \citeauthor{TGS82} lead to the correct
conclusion that some arguments of \citeauthor{TL81} are flawed, their
own counter-arguments are flawed due to a misunderstanding of
\citet{TL81}, \citet{Cha76}, or both. Even the direct response to
\citeauthor{TL81} from \citet{Cha81} is inadequate. Thus, the issue of
the fallacy of the averages lacks definitive closure and continues to
cause confusion in recent literature, as discussed in
\longref{sec:oft_short_term_rate}. We present the \citeauthor{Cha76}
model and the flawed criticism of it in \longref{sec:fa_charnov_model}.
We present the \citeauthor{Pulliam74} model and the flawed criticism of
it in \longref{sec:fa_pulliam_model}. We present issues with the
responses to these criticisms in \longref{sec:fa_responses}. 

\subsection{A Deterministic Foraging Model}
\label{sec:fa_charnov_model}

\Citet{Cha76} presents a ``completely deterministic''
\citep[p.~130]{Cha76} model for optimal foraging theory. The forager is
``assumed to make decisions as to maximize the net rate of energy intake
during a foraging bout'' \citep[p.~131]{Cha76}. The model analyzes a
forager searching for ``patches'' of which there are $n \in \N$ patch
types. The model calls $t$ the ``interpatch travel time,'' which
represents a deterministic time spent before encountering an additional
patch. For each $i \in \{1,2,\dots,n\}$, the model calls $P_i$ the
``proportion of the visited patches that are of type $i$'', $E_T$ the
``energy cost per unit time traveling between patches'', and $g_i(T_i)$
the  ``assimilated energy'' from handling a patch of type $i$ for a
total of $T_i$ time units. Therefore, if the agent encounters $N$
patches, $N P_i$ is the exact number (\ie, a deterministic quantity) of
patches of type $i$ and the total assimilated energy from all patches
divided by the total time spent searching for and handling all patches
is 
%
\begin{equation}
        \frac
        { \sum\limits_{i=1}^n N P_i g_i(T_i) - N t E_T }
        { N t + \sum\limits_{i=1}^n N P_i T_i }
        =
        \frac
        { \sum\limits_{i=1}^n P_i g_i(T_i) - t E_T }
        { t + \sum\limits_{i=1}^n P_i T_i }
        \label{eq:deterministic_net_rate}
\end{equation}
%
\Citeauthor{Cha76} calls this the \emph{net energy intake rate} for the
single bout. Note that while this rate looks similar in form to
$\E(\oft{G}_1)/\E(\oft{T}_1)$, this rate is \emph{not} meant to be an
expression of an expectation or even a quotient of expectations. This is
a ratio of two deterministic quantities representing total energy gained
over total time spent during an entire foraging bout. The only valid
criticism of \citeauthor{Cha76} might refer to some unnecessary uses of
the term \emph{average} in the description
\longref{eq:deterministic_net_rate}; however, the form of the expression
is definitely correct. In fact, the fallacy of the averages is
impossible in a deterministic context like this one.

\subsubsection{Criticism and the Per-Patch Rate}

In \citet{TL81}, all references to the $P_i$ from \citet{Cha76} are
replaced with $p_i$, which is defined by \citeauthor{TL81} as the
``probability of encountering patch type $i$.'' By substituting $P_i$
for $p_i$, they have made the mistake of turning a deterministic model
of foraging behavior into a stochastic one, which justifies the use of
probabilistic expectation. Their major claim is that \citeauthor{Cha76}
meant to use the ``rate of energy intake in patch type $i$'' and instead
use the ``average energy intake per patch divided by the average time
required to obtain the energy per patch.'' This claim is a result of a
fallacy of the averages. Therefore, with the caveat that interpatch
travel time $t$ is deterministic\footnote{\Citet{TL81} actually use the
term ``constant,'' which does not necessarily mean that $t$ is
deterministic; however, context implies that a constant and
deterministic $t$ is actually what is desired.}, \citeauthor{TL81} say
that the correct rate to use is
%
\begin{equation}
        \sum\limits_{i=1}^n p_i \frac{ g_i(T_i) - t E_T }{ t + T_i }
        \label{eq:tl_short_rate}
\end{equation}
%
If probability $p_i$ is replaced with the proportion $P_i$ from the
deterministic model of \citeauthor{Cha76} then this expression
represents the (arithmetic) mean \emph{per-patch} net energy intake
rate. However, \citet{Cha76} is very clear about optimizing net energy
intake over net time for an \emph{entire foraging bout} and not an
average per-patch rate. Thus, the criticisms that \citeauthor{TL81} make
of \citet{Cha76} are entirely due to a misunderstanding. 

\paragraph{Contradictory References:} \citet{TL81} also briefly
reference the stochastic derivations by \citet{Cha73} in order to
criticize another work. This implies that they simultaneously agree with
\citet{Cha73} and disagree with the similar work of \citet{Cha76}. This
is an interesting contradiction. A proper response to \citeauthor{TL81}
should use the model from \citeauthor{Cha73}. However, the prominent
responses all assume that \citeauthor{TL81} are criticizing
\citet{Cha73} and defend that work instead. Since \citeauthor{TL81}
clearly agree with \citeauthor{Cha73} then all such responses are
nonsense.

\paragraph{Use of Per-Patch Rate in Literature:} As we will discuss in
\longref{sec:oft_short_term_rate}, some authors have presented
qualitative observational justifications for optimizing a mean per-patch
net energy intake rate, and those authors have used
\longref{eq:tl_short_rate} as a template for such a rate. Interestingly,
these authors neglect that \citeauthor{TL81} state that $t$ must be
deterministic. In fact, if the interpatch travel time is a random
variable, using \longref{eq:tl_short_rate} to represent a mean per-patch
net energy intake is itself a fallacy of the averages. Thus, it is
extremely rare that the form of \longref{eq:tl_short_rate} has any value
in analysis of real systems.

\subsection{A Stochastic Foraging Model}
\label{sec:fa_pulliam_model}

\Citet{Pulliam74} presents a stochastic model of foraging that is almost
identical the one given by \citet{Cha73}. However, while
\citeauthor{Cha73} makes use of the theory of Poisson processes,
\citeauthor{Pulliam74} uses a notion of a ``random scatter of prey
items'' \citep[p.~62]{Pulliam74}. This notion generates something
similar to a Poisson process (\ie, it has the \emph{memorylessness}
property); however, it does not have the \emph{orderliness} property
that, among other things, forces simultaneous encounters to occur with
probability zero. Without this property, many of the arguments of
\citeauthor{Pulliam74} do not make rigorous mathematical sense. On the
other hand, the model assumes movement in such a way as to ``minimize
the probability of covering the same area twice''
\citep[p.~60]{Pulliam74} which could be argued to resemble orderliness
since prey are assumed to be stationary. Additionally, while some
language implies that limiting arguments are used, other language
implies that a limiting argument is not being used.  However, all of the
end results are correct from a limiting interpretation if the Poisson
assumption is made (\ie, orderliness is explicitly added to the existing
memorylessness assumption). Therefore, while the language of
\citeauthor{Pulliam74} can certainly be criticized, the end results
cannot. With a slight modification and an explicit limiting
interpretation, no fallacy of the averages has been committed. 

\subsubsection{Criticism and the Poisson Process}

\Citet{TL81} assume that \citet{Pulliam74} do not use a limiting
argument, and so they assume that a fallacy of averages has been
committed. In fact, they use the work of \citet{Cha73} to argue that the
argument of \citeauthor{Pulliam74} should only apply to large time
intervals (\ie, a limiting argument must be used). \Citeauthor{TL81}
then move to criticize the assumptions that \citet{Cha73} use to make
this argument true. In particular, they note that prey encounters must
be generated according to a Poisson process. However, \citeauthor{TL81}
misinterpret Poisson process properties when they incorrectly state that
encounters must be such that ``the probability of encountering a prey
item tends to zero on any finite interval as the number of prey items
tends to infinity'' which leads to them to state that ``prey items be
rather rare'' \citep[p.~392]{TL81}. This conclusion is their
justification for why Poisson encounters poorly model most real systems.
However, for Poisson encounters of prey items, probability of
encountering a prey item tends to zero on any finite zero as the number
of \emph{trials} tends to infinity. In other words, the probability of
encountering a prey item at any \emph{instant} of time is zero. This
assumption is very realistic; in fact, it would be strange for a model
of a real system not to assume this. It is more conventional to
criticize the independence of interarrival intervals and memorylessness
of interarrival times of a Poisson process, and it is strange that
\citeauthor{TL81} do not make this criticism considering that
\citeauthor{Pulliam74} do not make the Poisson assumption and instead
make the explicit assumption of independent arrivals and memorylessness.
Therefore, while \citet{Pulliam74} certainly deserves criticism for the
language used, the \citeauthor{TL81} criticism of \citeauthor{Pulliam74}
comes primarily from a misunderstanding of Poisson processes.

\subsection{The Responses}
\label{sec:fa_responses}

\Citet{GGP82} come to the defense of \citet{Cha76} by arguing that
\citet{TL81} have committed mathematical mistakes in their calculations.
Of course, the mistakes of \citeauthor{TL81} are in their understanding
and not in their calculations. In fact, the flawed argument of
\citeauthor{GGP82} itself is the result of a number of their own
mathematical and conceptual mistakes. \Citet{TGS82} defend both
\citet{Cha76} and \citet{Pulliam74}. However, their defense of
\citeauthor{Cha76} primarily supports arguments that are never
introduced by \citet{Cha76} nor \citet{Cha73}. Their defense of
\citet{Pulliam74} is incomplete and commits a number of small
mathematical mistakes. \Citet{Cha81} gives a direct response to
\citeauthor{TL81}. While this latter response is much shorter than the
others, it still manages to make same major mistake. We discuss the
problems with these responses below. Unfortunately, \citet{GGP82} and
\citet{TGS82} continue to be considered authoritative
\citep[\eg,{}][]{BK95,BW96,PHM90,REH90} despite having little analytical
value. In \longref{sec:oft_short_term_rate}, we present one example
where an entire body of research has its foundation in a
misunderstanding of \citet{TL81} and these inadequate responses to it.

\subsubsection{Fallacy of the Fallacy of the Traffic Policeman}

\Citet{GGP82} use the notation $p_i$ to represent the ``fraction of
patches that are of type $i$,'' which is similar to the $P_i$ introduced
by \citet{Cha76}; however, despite referring to the ``deterministic
character'' \citep[p.~877]{GGP82} of the \citeauthor{Cha76} model, they
introduce random variables for the ``energy gained ($G$) per patch
visited'' and the ``time ($T$) per visit'' and claim that
\longref{eq:deterministic_net_rate} is $\E(G)/\E(T)$
\citep[p.~875]{GGP82}. Of course, the \citeauthor{Cha76} model is
deterministic and the rate in \longref{eq:deterministic_net_rate} is
defined explicitly as the total net gain divided by the total time of an
entire foraging bout, and so \citeauthor{GGP82} are doubly mistaken.
They then call $t$ the ``average travel time between patches'' and state
that \longref{eq:tl_short_rate} represents $\E(G/T)$
\citep[p.~875]{GGP82}. This definition of $t$ unique from any of the
previous definitions and implies that travel time between patches must
be a random variable. However, if that's the case then $\E(G/T)$ equal
\longref{eq:tl_short_rate} in general; in fact, to say that these two
are equal commits another kind of fallacy of the averages\footnote{For
random variable $X$, $\E(1/X) \neq 1/\E(X)$. To say otherwise is false
in general.}. \Citet{GGP82} series of errors continues when they claim
that \citeauthor{TL81} call \longref{eq:tl_short_rate} an ``average rate
of energy intake'' \citep[p.~875]{GGP82}; in fact, \citeauthor{TL81}
call this the ``average value'' of the ``rate of energy intake in patch
type $i$'' \citep[pp.~390--391]{TL81}. Finally, \citeauthor{GGP82}
suggest that the formulation of $\E(G/T)$ given by \citeauthor{TL81}
contains a ``conceptual flaw'' \citep[p.~876]{GGP82} that can be fixed
by replacing each $p_i$ with $p_i'$, which is called ``the probability
of finding an animal in or traveling to patch type $i$ at a given
instant'' \citep[p.~877]{TL81} and defined as
%
\begin{equation*}
        p_i' = \frac{ p_i ( t + T_i ) }
                    { \sum\limits_{j=1}^n p_j ( t + T_j ) }
\end{equation*}
%
This substitution lets $\E(G/T)$ be expressed by
\longref{eq:deterministic_net_rate}. However, $p_i'$ is not a
probability. If it were such a probability then it should be used in the
expression for $\E(G)/\E(T)$ as well. In that case, $\E(G)/\E(T)$ would
\emph{not} match \longref{eq:deterministic_net_rate} as it must in their
argument. Therefore, \citeauthor{GGP82} present a confusing,
contradictory, and counter-productive defense of \citeauthor{Cha76}.

\subsubsection{Fallacy of the Fallacy of the Fallacy of the Averages}

In \citet{TGS82}, it is correctly noted that \citet{Cha76} states that
the quantity to be maximized is the net rate of energy intake ``during a
foraging bout.'' However, much of the rest of their argument concerns
stochastic arguments that make no sense for the deterministic model of
\citeauthor{Cha76}. While it is possible that they mean to refer to
\citet{Cha73}, the particular stochastic limits used by
\citeauthor{TGS82} are never discussed by \citeauthor{Cha73}.
Additionally, it makes little sense to claim that \citeauthor{Cha73}
answers \citeauthor{TL81} since \citeauthor{TL81} use these same
arguments in their criticism of \citet{Pulliam74}. \Citeauthor{TGS82}
also reference \citet{GGP82} as giving ``the correct analysis of the
(short-term) average rate of energy intake'' \citep[p.~883]{TGS82} which
has already been shown to also be nonsensical, and so \citeauthor{TGS82}
inherit all of the previous objections to the argument of
\citeauthor{GGP82}.

\citeauthor{TGS82} defend \citet{Pulliam74} for the same reasons that we
do. In particular, they point out that \citeauthor{TL81} misunderstand
the mathematical description of a continuous-time Poisson counting
process. However, \citeauthor{TGS82} introduce a quantity analogous to
our $\oft{T}^N$ and state that $\E(N/\oft{T}^N)$ converges to
$1/\E(\oft{T}_1)$ as $N \to \infty$ in general. As shown by
\citet{JohnsMiller63}, this is only true when there exists some $K \in
\N$ with $\E(1/\oft{T}^K) < \infty$. \Citeauthor{TGS82} continue this
discussion of convergence by giving the Taylor series expansion
%
\begin{equation*}
        \E\left( \frac{N}{\oft{T}^N} \right)
        \approx 
        \frac{1}{\E(\oft{T}_1)} 
        + 
        \frac{\var(\oft{T}_1)}{N \E(\oft{T}_1)^3}
\end{equation*}
%
While this is certainly a valid expansion, they claim that it is the
result of expanding $1/\oft{T}^N$ about $\E(\oft{T}_1)$. In fact, the
expansion given above is the result of expadnding $1/\oft{T}^N$ about
$\E(\oft{T}^N)$ (\ie, $N \E(\oft{T}_1)$). These two expansions will only
be similar for small $N$. The purpose of this expansion is to justify
approximating $\E( N/\oft{T}^N )$ by $1/\E(\oft{T}_1)$ as the
approximation error will be small provided $N$ is sufficiently large.
However, as there is no similar Taylor series expansion for
$\E(\oft{G}^N/\oft{T}^N)$ because $\oft{G}^N$ and $\oft{T}^N$ will
covary in general\footnote{However, the covariance of these two random
variables should decrease quickly with increasing $N$.}, this is not a
helpful argument for justifying the fallacy of the averages. 

\subsubsection{A Direct Response}

Interestingly, a direct answer to \citet{TL81} is given by \citet{Cha81}
as a single paragraph. It states that \citeauthor{TL81} ``have chosen to
ignore the fact that cumulative renewal theory \emph{was} the basis for
using'' \longref{eq:tl_short_rate} as an average rate of energy intake.
It states that this ``is clearly stated in'' \citet{Cha76} and ``well
developed'' by \citet{Cha73}. However, as already discussed, the former
presents a ``completely deterministic'' model. Additionally, the latter
only uses \longref{eq:tl_short_rate} when describing deterministic
foraging models. Its treatment of stochastic models abandons the $P_i$
notation for split Poisson process probabilities $\lambda_i / \lambda$.
That is, the cumulative renewal theory results by \citet{Cha73} use
notation similar to what is used in our stochastic foraging model.
Thus, even \citet{Cha81} fails to provide an adequate response to
\citet{TL81}.

\section{The Short-Term Rate}
\label{sec:oft_short_term_rate}

\todo{Some biologists have argued from empirical data that $\E(G/T)$
should be used instead of $\E(G)/E(T)$. I show that the empirical data
does NOT imply this and that their expressions for $\E(G/T)$ are
incorrect. I need to include these results here.}

%then this expression would be similar to our $\E( \oft{G}_1 /
%\oft{T}_1 )$, which has commonly come to be known as the
%\emph{short-term rate} \citep{REH90,Real91,TGS82}, the \emph{per-patch
%rate} \citep{SK86}, or the \emph{\acro{EoR}{expectation of rates}}
%\citep{BK96,BW96}. However, there is nothing by \citet{Cha73} nor
%\citet{Cha76} indicating that \citeauthor{Cha76} intended on maximizing
%this short-term rate. So, again, the arguments of \citeauthor{TL81} are
%simply due to a misreading of \citet{Cha73} and \citet{Cha76}.

\section{On-line Estimation of Long-Term Rate}
\label{sec:oft_est_long_term_rate}

\todo{There is some confusion in the literature on how to estimate
long-term rate on-line. Few realize that the RATE of POINT GAIN is not
what is actually being actively estimated and maintained but rather then
ENCOUNTER RATE. In this section, I show two methods of estimating the
encounter rate and give stochastic results that show that both
estimation techniques will converge to the actual encounter rate with
probability 1. This section actually relates to
\longref{sec:oft_short_term_rate} because much of the discussion of
estimation of rates in the literature has been done by advocates of the
short-term rate approach.}

The extensive use of the expression $\E(\oft{G}_1)/\E(\oft{T}_1)$ in
foraging literature without reference to what it represents has led to
some strange conclusions. 

Observational evidence \citep[\eg,][]{BK95,HR87} has suggested that some
foragers behave in a way that optimizes
%
\begin{align}
        \E\left(\frac{\oft{G}_1}{\oft{T}_1}\right)
        \label{eq:oft_payoff_EoR}
\end{align}
%
By itself, it is not clear how the maximization of this expression would
capture any important measures of Darwinian fitness \citep{GGP82,TGS82}.

However,  
$\E(\oft{G}_1)/\E(\oft{T}_1)$ is not necessarily


Whereas the long-term rate of gain seems to encapsulate important
measures of Darwinian fitness, \longref{eq:oft_payoff_EoR} alone does
not appear to measure  

\citep{GGP82,TGS82}

We stress that the expression
$\E(\oft{G}_1)/\E(\oft{T}_1)$ is meant to be an
analytical tool for understanding the impact of the structure of more
fundamental stochastic rates $\oft{G}/\oft{T}$ and $\oft{G}/t$ on
behavior. If an agent is designed to monitor is long-term rate of gain
as part of an algorithm to maximize its performance in an unknown
environment, it need not estimate the expectations
$\E(\oft{G}_1)$ and $\E(\oft{T}_1)$. Instead, it only
needs to calculate $\oft{G}/\oft{T}$ or $\oft{G}/t$. The former
expression requires storage of two sums which can be updated at each new
encounter and the calculation of a quotient of those sums. The latter
expression requires storage of a single sum which can be updated at each
new encounter and the calculation of the quotient of that sum with the
agent's runtime, which may be inferred from internal (\eg, a clock) or
external (\eg, the sun) cues. Thus, the latter expression in particular
requires relatively low computational burden.

We do not wish to imply that it is necessary for foragers to have the
ability to calculate their long-term rate of gain on-line. If the
long-term rate of gain truly is a surrogate for Darwinian fitness,
agents that have heritable behaviors that happen to maximize long-term
rate of gain will be favored by natural selection. That is, being able
to perform calculations about the environment in order to adapt to it
will provide no survival advantage to foragers in a relatively static
environment. Thus, in these static environments, classical \ac{OFT} does
not predict \emph{how} foragers are making decisions but rather
\emph{why} certain decisions have been favored by the natural selection
accompanying a certain environment. However, there is some observational
evidence \citep[\eg,][]{BK95,HR87} that deviates from the expected
optimum behaviors predicted by classical \ac{OFT} (\ie, behaviors that
do not maximize long-term rate of gain). This has motivated some
behavioral ecologists to suggest that agents do perform quantitative
calculations internally \citep[\eg,][]{BK95,BK96,BW96,REH90}. These
ecologists claim that limitations in memory or computational ability
prevent some foragers from estimating $\E(\oft{G}_1)$ and
$\E(\oft{T}_1)$ accurately and so those foragers either use bad
estimates of long-term rate of gain

Observational evidence \citep[\eg,][]{BK95,HR87} has suggested that some
foragers have behaviors that happen to optimize
%
\begin{align}
        \E\left(\frac{\oft{G}_1}{\oft{T}_1}\right)
        \label{eq:oft_payoff_EoR_old}
\end{align}
%
where $k \in \N$ is an arbitrary foraging cycle. In an effort to explain
these deviations from classical \ac{OFT} (\ie, optimizing
$\E(\oft{G}_1)/\E(\oft{T}_1)$), a number of papers \citep[in
particular,][]{BK95,BK96,BW96,REH90} suggest that foragers actively
evaluate performance in real-time by calculating payoff function
estimates internally and adjust behavior in order to maximize those
internal metrics. From this assumption, the papers conclude that if
$\E(\oft{G}_1)/\E(\oft{T}_1)$ is too complex to calculate, the foragers
will either use a very bad estimate of $\E(\oft{G}_1)/\E(\oft{T}_1)$
that leads to suboptimal behavior or will pick some other payoff
function that is simple to calculate and yet also leads to beneficial
behaviors. \Citet{REH90} states that there ``may not be a single
currency that is most appropriate for all biological systems'' and that
``there may be a most appropriate method for processing information,
conditional upon the organism's memory capabilities.'' The papers claim
that $\E(\oft{G}_1)/\E(\oft{T}_1)$ would be estimated by storing in
memory each individual gain $\oft{G}_{\oft{j}}$ and cycle time
$\oft{T}_{\oft{j}}$ for all $j \in \{1,\dots,N\}$ and calculating the
ratio of sums $\oft{G}/\oft{T}$, where $N$ is the total number of
foraging cycles completed. By \longref{eq:oft_payoff_long_rate},
$\oft{G}/\oft{T}$ will converge with probability 1 to
$\E(\oft{G}_1)/\E(\oft{T}_1)$ as $N$ gets large. However, these papers
suggest that storing those $2N$ values for a large $N$ would be too
complex for certain foragers. Thus, other payoff functions with ``no
obvious functional advantage'' \citep{BK96} are defined that require
less storage and also lead to behaviors that optimize
\longref{eq:oft_payoff_EoR}. We note that $\E(\oft{G}_1)/\E(\oft{T}_1)$
can be estimated without storing in memory a large number of previous
net gains and cycle times.  In fact, if the sums $\oft{G}$ and $\oft{T}$
are kept in memory after $N$ cycles, they can be updated after cycle
$N+1$ by adding $\oft{G}_{N+1}$ and $\oft{T}_{N+1}$ respectively. In
fact, if only the sum $\oft{G}$ is stored in memory and lifetime $t$ is
inferred from environmental or physiological cues,
\longref{eq:oft_payoff_long_rate} shows that the ratio $\oft{G}/t$ will
also converge to $\E(\oft{G}_1)/\E(\oft{T}_1)$.  Additionally, while it
may be possible that foragers are calculating these rates in real-time,
it seems more likely that natural selection will choose foragers that
have behaviors that happen to maximize $\E(\oft{G}_1)/\E(\oft{T}_1)$ for
their relatively static environment. 

\section{Simultaneous Encounters}
\label{sec:oft_simultaneous_encounters}

\todo{Many authors (especially those mentioned in
\longref{sec:oft_short_term_rate}) use experiments with simultaneous
encounters and claim that the results invalidate the use of the
long-term rate. I show that these, in fact, do not invalidate the use of
the long-term rate. I stress that one of the key assumptions of OFT is
that simultaneous encounters occur with probability 0 and any
experimental design that does not meet this requirement simply does not
apply to OFT. Many authors forget that Poisson encounters require more
than just memorylessness; orderliness is important.}


