Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
149 commits
Select commit Hold shift + click to select a range
eaadcbc
WIP: PeriodArray
TomAugspurger Sep 26, 2018
a05928a
WIP
TomAugspurger Sep 27, 2018
3c0d9ee
Just moves
TomAugspurger Sep 27, 2018
63fc3fa
PeriodArray.shift definition
TomAugspurger Sep 27, 2018
7d5d71c
_data type
TomAugspurger Sep 27, 2018
e5caac6
clean
TomAugspurger Sep 27, 2018
c194407
accessor wip
TomAugspurger Sep 27, 2018
eb4506b
some more wip
TomAugspurger Sep 27, 2018
1b9fd7a
tshift, shift
TomAugspurger Sep 28, 2018
0fa0ed1
Arithmetic
TomAugspurger Sep 28, 2018
3247ea8
repr changes
TomAugspurger Sep 28, 2018
c162cdd
wip
TomAugspurger Sep 28, 2018
611d378
freq setter
TomAugspurger Sep 28, 2018
fb2ff82
Added disabled ops
TomAugspurger Sep 28, 2018
25a380f
copy
TomAugspurger Sep 28, 2018
1b2c4ec
Support concat
TomAugspurger Sep 28, 2018
d04293e
object ctor
TomAugspurger Sep 28, 2018
eacad39
Updates
TomAugspurger Sep 28, 2018
70cd3b8
lint
TomAugspurger Sep 28, 2018
9b22889
lint
TomAugspurger Sep 28, 2018
87d289a
Merge remote-tracking branch 'upstream/master' into ea-period
TomAugspurger Oct 1, 2018
6369c7f
wip
TomAugspurger Oct 1, 2018
01551f0
more wip
TomAugspurger Oct 1, 2018
0437940
array-setitem
TomAugspurger Oct 1, 2018
42ab137
wip
TomAugspurger Oct 1, 2018
298390f
wip
TomAugspurger Oct 1, 2018
23e5cfc
Use ._tshift internally for datetimelike ops
TomAugspurger Oct 2, 2018
9d17fd2
deep
TomAugspurger Oct 2, 2018
959cd72
Squashed commit of the following:
TomAugspurger Oct 2, 2018
b66f617
Squashed commit of the following:
TomAugspurger Oct 2, 2018
5669675
fixup
TomAugspurger Oct 2, 2018
2c0311c
The rest of the EA tests
TomAugspurger Oct 2, 2018
012be1c
docs
TomAugspurger Oct 2, 2018
c3a96d0
Merge remote-tracking branch 'upstream/master' into datetimelike-tshift
TomAugspurger Oct 3, 2018
67faabc
rename to time_shift
TomAugspurger Oct 3, 2018
ff7c06c
Squashed commit of the following:
TomAugspurger Oct 3, 2018
c2d57bd
Squashed commit of the following:
TomAugspurger Oct 3, 2018
fbde770
Squashed commit of the following:
TomAugspurger Oct 3, 2018
1c4bbe7
Squashed commit of the following:
TomAugspurger Oct 3, 2018
b395c90
fixed merge conflict
TomAugspurger Oct 3, 2018
d68a5c5
Handle divmod test
TomAugspurger Oct 3, 2018
0c7b704
extension tests passing
TomAugspurger Oct 3, 2018
d26d3d2
Squashed commit of the following:
TomAugspurger Oct 4, 2018
e4babea
Merge remote-tracking branch 'upstream/master' into ea-period
TomAugspurger Oct 4, 2018
7f6c144
merge conflict
TomAugspurger Oct 4, 2018
b4aa4ca
wip
TomAugspurger Oct 4, 2018
6a70131
indexes passing
TomAugspurger Oct 4, 2018
9aa077c
op names
TomAugspurger Oct 4, 2018
411738c
extension, arrays passing
TomAugspurger Oct 4, 2018
8e0fb69
Merge remote-tracking branch 'upstream/master' into ea-period
TomAugspurger Oct 9, 2018
6d98e85
fixup
TomAugspurger Oct 9, 2018
6d9e150
lint
TomAugspurger Oct 9, 2018
4899479
Fixed to_timestamp
TomAugspurger Oct 9, 2018
634def1
Same error message for index, series
TomAugspurger Oct 9, 2018
1f18452
Fix freq handling in to_timestamp
TomAugspurger Oct 9, 2018
2f92b22
dtype update
TomAugspurger Oct 9, 2018
23f232c
accept kwargs
TomAugspurger Oct 9, 2018
dd3b8cd
fixups
TomAugspurger Oct 9, 2018
1a7c360
Merge remote-tracking branch 'upstream/master' into ea-period
TomAugspurger Oct 9, 2018
87ecb64
updates
TomAugspurger Oct 9, 2018
0bde329
explicit
TomAugspurger Oct 9, 2018
2d85a82
add to assert
TomAugspurger Oct 9, 2018
438e6b5
wip period_array
TomAugspurger Oct 10, 2018
a9456fd
Merge remote-tracking branch 'upstream/master' into ea-period
TomAugspurger Oct 10, 2018
ac05365
wip period_array
TomAugspurger Oct 10, 2018
36ed547
order
TomAugspurger Oct 10, 2018
4652ca7
sort order
TomAugspurger Oct 10, 2018
a047a1b
test for hashing
TomAugspurger Oct 10, 2018
a4a30d7
update
TomAugspurger Oct 10, 2018
1441ae6
lint
TomAugspurger Oct 10, 2018
8003808
boxing
TomAugspurger Oct 10, 2018
5f43753
fix fixtures
TomAugspurger Oct 10, 2018
1c13d0f
infer
TomAugspurger Oct 10, 2018
bae6b3d
Remove seemingly unreachable code
TomAugspurger Oct 10, 2018
f422cf0
lint
TomAugspurger Oct 10, 2018
0229d74
wip
TomAugspurger Oct 12, 2018
aa40cf4
Merge remote-tracking branch 'upstream/master' into ea-period
TomAugspurger Oct 12, 2018
29085e1
Updates for master
TomAugspurger Oct 12, 2018
00ffddf
simplify
TomAugspurger Oct 12, 2018
e81fa9c
wip
TomAugspurger Oct 12, 2018
0c8925f
Merge remote-tracking branch 'upstream/master' into ea-period
TomAugspurger Oct 15, 2018
96204a1
remove view
TomAugspurger Oct 15, 2018
82930f7
Merge remote-tracking branch 'upstream/master' into ea-period
TomAugspurger Oct 17, 2018
8d24582
simplify
TomAugspurger Oct 17, 2018
1fc7744
lint
TomAugspurger Oct 17, 2018
6cd428c
Removed add_comparison_methods
TomAugspurger Oct 17, 2018
21693e0
xfail op
TomAugspurger Oct 17, 2018
b65ffad
remove some
TomAugspurger Oct 17, 2018
1f438e3
constructors
TomAugspurger Oct 17, 2018
f3928fb
Constructor cleanup
TomAugspurger Oct 17, 2018
089f8ab
misc fixups
TomAugspurger Oct 17, 2018
700650a
more xfails
TomAugspurger Oct 17, 2018
452c229
typo
TomAugspurger Oct 17, 2018
e3e0e57
Merge remote-tracking branch 'upstream/master' into ea-period
TomAugspurger Oct 18, 2018
78751c2
Added asi8
TomAugspurger Oct 18, 2018
203d561
Allow setting nan
TomAugspurger Oct 18, 2018
eb1c67d
revert breaking docs
TomAugspurger Oct 18, 2018
e08aa79
Override _add_sub_int_array
TomAugspurger Oct 18, 2018
c1ee04b
lint
TomAugspurger Oct 18, 2018
827e563
Update PeriodIndex._simple_new
TomAugspurger Oct 18, 2018
ca4a7fd
Clean up uses of .values, ._values, ._ndarray_values, ._data
TomAugspurger Oct 18, 2018
ed185c0
one more values
TomAugspurger Oct 18, 2018
b3407ac
Merge remote-tracking branch 'upstream/master' into ea-period
TomAugspurger Oct 18, 2018
a4011eb
remove xfails
TomAugspurger Oct 18, 2018
fc1ca3c
Fixed freq handling in _shallow_copy with a freq
TomAugspurger Oct 18, 2018
1b1841f
test updates
TomAugspurger Oct 18, 2018
b3b315a
API: Keep PeriodIndex.values an ndarray
TomAugspurger Oct 18, 2018
3ab4176
Merge remote-tracking branch 'upstream/master' into ea-period
TomAugspurger Oct 18, 2018
8102475
BUG: Raise for non-equal freq in take
TomAugspurger Oct 18, 2018
8c329eb
Punt on DataFrame.replace specializing
TomAugspurger Oct 18, 2018
78d4960
lint
TomAugspurger Oct 18, 2018
4e3d914
fixed xfail message
TomAugspurger Oct 18, 2018
5e4aaa7
TST: _from_datetime64
TomAugspurger Oct 19, 2018
7f77563
Fixups
TomAugspurger Oct 19, 2018
f88d6f7
escape
TomAugspurger Oct 19, 2018
7aa78ba
dtype
TomAugspurger Oct 19, 2018
2d737f8
revert and unxfail values
TomAugspurger Oct 19, 2018
833899a
error catching
TomAugspurger Oct 19, 2018
236b49c
isort
TomAugspurger Oct 19, 2018
8230347
Avoid PeriodArray.values
TomAugspurger Oct 19, 2018
bf33a57
clarify _box_func usage
TomAugspurger Oct 19, 2018
738acfe
Merge remote-tracking branch 'upstream/master' into ea-period
TomAugspurger Oct 19, 2018
032ec02
TST: unxfail ops tests
TomAugspurger Oct 19, 2018
77e389a
Avoid use of .values
jorisvandenbossche Oct 19, 2018
61031d7
__setitem__ type
TomAugspurger Oct 19, 2018
a094b3d
Misc cleanups
TomAugspurger Oct 19, 2018
ace4856
lint
TomAugspurger Oct 19, 2018
fc6a1c7
API: remove ordinal from period_array
TomAugspurger Oct 19, 2018
900afcf
catch exception
TomAugspurger Oct 19, 2018
0baa3e9
misc cleanup
TomAugspurger Oct 19, 2018
f95106e
Handle astype integer size
TomAugspurger Oct 19, 2018
e57e24a
Bump test coverage
TomAugspurger Oct 19, 2018
ce1c970
remove partial test
TomAugspurger Oct 19, 2018
a7e1216
close bracket
TomAugspurger Oct 19, 2018
2548d6a
change the test
TomAugspurger Oct 19, 2018
02e3863
isort
TomAugspurger Oct 19, 2018
1997cff
consistent _data
TomAugspurger Oct 19, 2018
af2d1de
lint
TomAugspurger Oct 19, 2018
64f5778
Merge remote-tracking branch 'upstream/master' into ea-period
TomAugspurger Oct 20, 2018
4151510
ndarray_values -> asi8
TomAugspurger Oct 20, 2018
ac9bd41
colocate ops
TomAugspurger Oct 20, 2018
5462bd7
refactor PeriodIndex.item
TomAugspurger Oct 20, 2018
c1c6428
return NotImplemented for Series / Index
TomAugspurger Oct 20, 2018
7ab2736
remove xpass
TomAugspurger Oct 20, 2018
bd6f966
release note
TomAugspurger Oct 22, 2018
8068daf
Merge remote-tracking branch 'upstream/master' into ea-period
TomAugspurger Oct 23, 2018
5691506
types, use data
TomAugspurger Oct 23, 2018
575d61a
remove ufunc xpass
TomAugspurger Oct 24, 2018
4065bdb
Merge remote-tracking branch 'upstream/master' into ea-period
TomAugspurger Oct 25, 2018
File filter

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
tshift, shift
  • Loading branch information
TomAugspurger committed Sep 28, 2018
commit 1b9fd7aa0018b7e615c58bbe9b98228171978570
10 changes: 3 additions & 7 deletions pandas/core/accessor.py
Original file line number Diff line number Diff line change
Expand Up @@ -68,8 +68,6 @@ def _add_delegate_accessors(cls, delegate, accessors, typ,
overwrite : boolean, default False
overwrite the method/property in the target class if it exists
"""
delegated_to = getattr(delegate, '_delegated_to', delegate)

def _create_delegator_property(name):

def _getter(self):
Expand All @@ -82,15 +80,15 @@ def _setter(self, new_values):
_setter.__name__ = name

return property(fget=_getter, fset=_setter,
doc=getattr(delegated_to, name).__doc__)
doc=getattr(delegate, name).__doc__)

def _create_delegator_method(name):

def f(self, *args, **kwargs):
return self._delegate_method(name, *args, **kwargs)

f.__name__ = name
f.__doc__ = getattr(delegated_to, name).__doc__
f.__doc__ = getattr(delegate, name).__doc__

return f

Expand All @@ -107,12 +105,10 @@ def f(self, *args, **kwargs):


def delegate_names(delegate, accessors, typ, overwrite=False):
# type (type, list, str, bool) -> Callable
"""
Add delegated names to a class using a class decorator.

Methods and attributes are delegated to ...


This provides an alternative usage to directly calling
`_add_delegate_accessors` below a class definition.

Expand Down
7 changes: 6 additions & 1 deletion pandas/core/arrays/datetimelike.py
Original file line number Diff line number Diff line change
Expand Up @@ -553,6 +553,9 @@ def shift(self, periods, freq=None):
--------
Index.shift : Shift values of Index.
"""
return self._tshift(periods, freq=freq)

def _tshift(self, periods, freq=None):
if freq is not None and freq != self.freq:
if isinstance(freq, compat.string_types):
freq = frequencies.to_offset(freq)
Expand Down Expand Up @@ -600,7 +603,9 @@ def __add__(self, other):
elif lib.is_integer(other):
# This check must come after the check for np.timedelta64
# as is_integer returns True for these
result = self.shift(other)
# TODO: make a _shift method that's consistent between
# Index and EA
result = self._tshift(other)

# array-like others
elif is_timedelta64_dtype(other):
Expand Down
42 changes: 41 additions & 1 deletion pandas/core/arrays/period.py
Original file line number Diff line number Diff line change
Expand Up @@ -113,9 +113,32 @@ class PeriodArray(DatetimeLikeArrayMixin, ExtensionArray):
"""
_attributes = ["freq"]
_typ = "period" # ABCPeriodArray

# Names others delegate to us on
_other_ops = []
_bool_ops = ['is_leap_year']
_object_ops = ['start_time', 'end_time', 'freq']
_field_ops = ['year', 'month', 'day', 'hour', 'minute', 'second',
'weekofyear', 'weekday', 'week', 'dayofweek',
'dayofyear', 'quarter', 'qyear',
'days_in_month', 'daysinmonth']
_datetimelike_ops = _field_ops + _object_ops + _bool_ops
_datetimelike_methods = ['strftime', 'to_timestamp', 'asfreq']

# --------------------------------------------------------------------
# Constructors

@property
def _foo(self):
return 'foo!'

@_foo.setter
def _foo(self, value):
print("setting foo to ", value)

def _bar(self, arg, kwarg=1):
print(arg, kwarg)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These go, right?


def __new__(cls, data=None, ordinal=None, freq=None, start=None, end=None,
periods=None, tz=None, dtype=None, copy=False,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tz very likely doesnt belong

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Part of why the constructors for the existing mixins are bare-bones is because there are comments in the Index subclasses suggesting things like start/end should be taken out of them.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I mentioned on the mailing list, I would rather go for a very simple constructor (basically what _simple_new is now below). I don't think our array classes should have a __new__.
If it is too much work for this PR to refactor this __new__ method, we could also leave it for now as is but under another name (eg _complex_new :-))

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And I think we could then combine _simple_new and _from _ordinals ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To maybe clarify what I mean, a small change to IntervalArray: https://github.com/pandas-dev/pandas/compare/master...jorisvandenbossche:intervalarray?expand=1 (needs better naming of course): remove fastpath (should be done anyway as it is not used), and use __init__ instead of __new__. Of course, the functionality of passing a single array-like (of scalars or of interval dtype) what is now in the __new__ could still be kept in the __init__ if we find that important.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No preference on doing it as part of this PR or another. The diff is already impossible to read, so I may as well try doing it here.

**fields):
Expand Down Expand Up @@ -511,11 +534,27 @@ def shift(self, periods=1):
-------
shifted : Period Array/Index
"""
# We have two kinds of shift.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit ugly, but I think unavoidable.

# 1. ExtensionArray.shift: move positions of each value,
# fill NA on the end
# 2. Datelike.tshift: move each value through time
# Each Datelike array will implement both. It's up to the
# caller to call the correct one.
return self._ea_shift(periods=periods)

def _ea_shift(self, periods=1):
# TODO: remove from DatetimeLikeArrayMixin
# The semantics for Index.shift differ from EA.shift
# then just call super.
return ExtensionArray.shift(self, periods)

def _tshift(self, n, freq=None):
# TODO: docs
values = self.values + n * self.freq.n
if self.hasnans:
values[self._isnan] = iNaT
return self._simple_new(values, freq=self.freq)

def _maybe_convert_timedelta(self, other):
"""
Convert timedelta-like input to an integer multiple of self.freq
Expand Down Expand Up @@ -628,7 +667,8 @@ def end_time(self):

PeriodArray._add_comparison_ops()
PeriodArray._add_datetimelike_methods()

# PeriodArray._add_numeric_methods_disabled()
# PeriodArray._add_logical_methods_disabled()

# -------------------------------------------------------------------
# Constructor Helpers
Expand Down
10 changes: 4 additions & 6 deletions pandas/core/indexes/accessors.py
Original file line number Diff line number Diff line change
Expand Up @@ -270,11 +270,11 @@ def freq(self):
return self._get_values().inferred_freq


@delegate_names(delegate=PeriodIndex,
accessors=PeriodIndex._datetimelike_ops,
@delegate_names(delegate=PeriodArray,
accessors=PeriodArray._datetimelike_ops,
typ="property")
@delegate_names(delegate=PeriodIndex,
accessors=PeriodIndex._datetimelike_methods,
@delegate_names(delegate=PeriodArray,
accessors=PeriodArray._datetimelike_methods,
typ="method")
class PeriodProperties(Properties):
"""
Expand All @@ -289,8 +289,6 @@ class PeriodProperties(Properties):
Returns a Series indexed like the original Series.
Raises TypeError if the Series does not contain datetimelike values.
"""
def _delegate_method(self, name, *args, **kwargs):
pass


class CombinedDatetimelikeProperties(DatetimeProperties, TimedeltaProperties):
Expand Down
40 changes: 21 additions & 19 deletions pandas/core/indexes/period.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# pylint: disable=E1101,E1103,W0232
from datetime import datetime
import numpy as np
import operator
import warnings

from pandas.core import common as com
Expand All @@ -20,6 +21,7 @@
import pandas.tseries.frequencies as frequencies
from pandas.tseries.frequencies import get_freq_code as _gfc

from pandas.core.accessor import PandasDelegate, delegate_names
from pandas.core.indexes.datetimes import DatetimeIndex, Int64Index, Index
from pandas.core.indexes.datetimelike import DatelikeOps, DatetimeIndexOpsMixin
from pandas.core.tools.datetimes import parse_time_string
Expand Down Expand Up @@ -64,8 +66,23 @@ def _new_PeriodIndex(cls, **d):
return cls._from_ordinals(values=values, **d)


class PeriodDelegateMixin(PandasDelegate):
"""
Delegate from PeriodIndex to PeriodArray.
"""
def _delegate_property_get(self, name, *args, **kwargs):
return getattr(self._data, name)

def _delegate_property_set(self, name, value, *args, **kwargs):
setattr(self._data, name, value)

def _delegate_method(self, name, *args, **kwargs):
return operator.methodcaller(name, *args, **kwargs)(self._data)


# @delegate_names(PeriodArray, PeriodArray._datetimelike_methods, typ="method")
class PeriodIndex(DatelikeOps, DatetimeIndexOpsMixin,
Int64Index):
Int64Index, PeriodDelegateMixin):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are people OK with PeriodIndex also being an accessor? That seemed like the easiest way to dispatch things down to PeriodArray.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is performance affected? I'm still in the "inheritance is the easiest way to dispatch" camp, but recognize that I've lost this one.

Copy link
Contributor Author

@TomAugspurger TomAugspurger Oct 12, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beyond an extra function call (or two), I don't think so, though I will check. These aren't descriptors so we aren't changing getattribute.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could make a generic mechanism to do this at some point. We are eventually going to be doing this with all Index.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Related to that, as I think this is for a follow-up, how do we keep track of the follow-up ideas? Update a list in the top-post (to not have it somewhere in the middle be hidden by github)

"""
Immutable ndarray holding ordinal values indicating regular periods in
time such as particular years, quarters, months, etc.
Expand Down Expand Up @@ -145,19 +162,9 @@ class PeriodIndex(DatelikeOps, DatetimeIndexOpsMixin,
"""
_typ = 'periodindex'
_attributes = ['name', 'freq']
_delegated_to = PeriodArray
# _delegated_to = PeriodArray

# define my properties & methods for delegation
_other_ops = []
_bool_ops = ['is_leap_year']
_object_ops = ['start_time', 'end_time', 'freq']
_field_ops = ['year', 'month', 'day', 'hour', 'minute', 'second',
'weekofyear', 'weekday', 'week', 'dayofweek',
'dayofyear', 'quarter', 'qyear',
'days_in_month', 'daysinmonth']
_datetimelike_ops = _field_ops + _object_ops + _bool_ops
_datetimelike_methods = ['strftime', 'to_timestamp', 'asfreq']

_is_numeric_dtype = False
_infer_as_myclass = True

Expand Down Expand Up @@ -339,13 +346,8 @@ def shift(self, n):
-------
shifted : PeriodIndex
"""
# TODO: docs
# Note, this differs from the definition of ExtensionArray.shift
# so we don't dispatch
values = self._ndarray_values + n * self.freq.n
if self.hasnans:
values[self._isnan] = tslib.iNaT
return self._shallow_copy(values=values)
i8values = self._data._tshift(n)
return self._simple_new(i8values, name=self.name, freq=self.freq)

def _coerce_scalar_to_index(self, item):
"""
Expand Down