Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Volume = 0 #1935

Open
jvallsgm opened this issue May 15, 2024 · 1 comment
Open

Volume = 0 #1935

jvallsgm opened this issue May 15, 2024 · 1 comment

Comments

@jvallsgm
Copy link

Describe bug

Hello,
in the 1 or 5 minute history, for example of 'BTC-USD' often the volume is 0. There seems to be no negative volume at all. Could it be that it is failing volume and whenever it has to be negative a 0 is stored?
I have tried both with 'repair' to True or False.
Thanks.

Simple code that reproduces your problem

import yfinance as yf
stock = yf.Ticker('BTC-USD')
historical_prices = stock.history(start='2024-05-09', end='2024-05-10', interval='5m', repair=True)

Debug log

DEBUG Entering history()
DEBUG Entering history()
DEBUG BTC-USD: Yahoo GET parameters: {'period1': '2024-05-09 00:00:00+00:00', 'period2': '2024-05-10 00:00:00+00:00', 'interval': '5m', 'includePrePost': False, 'events': 'div,splits,capitalGains'}
DEBUG BTC-USD: yfinance received OHLC data: 2024-05-09 00:00:00 -> 2024-05-09 23:55:00
DEBUG BTC-USD: OHLC after cleaning: 2024-05-09 00:00:00+00:00 -> 2024-05-09 23:55:00+00:00
DEBUG BTC-USD: OHLC after combining events: 2024-05-09 00:00:00+00:00 -> 2024-05-09 23:55:00+00:00
DEBUG BTC-USD: checking OHLC for repairs ...
DEBUG Entering _fix_unit_mixups()
DEBUG Entering _fix_unit_switch()
DEBUG Entering _fix_prices_sudden_change()
DEBUG price-repair-split: appears_suspended=False, idx_latest_active=0 (2024-05-09)
INFO price-repair-split: No 100x errors detected
DEBUG Exiting _fix_prices_sudden_change()
DEBUG Exiting _fix_unit_switch()
DEBUG Entering _fix_unit_random_mixups()
INFO price-repair-100x: No sporadic 100x errors
DEBUG Exiting _fix_unit_random_mixups()
DEBUG Exiting _fix_unit_mixups()
DEBUG Entering _fix_bad_stock_split()
DEBUG Exiting _fix_bad_stock_split()
DEBUG Entering _fix_zeroes()
DEBUG Entering _reconstruct_intervals_batch()
DEBUG min_dt=2024-03-18 00:00:00+00:00 interval=5m sub_interval=2m
DEBUG grp_max_size = 30 days, 0:00:00
DEBUG Repair groups:
DEBUG - 2024-05-09 00:00:00+00:00 -> 2024-05-09 23:45:00+00:00
DEBUG df_block:
Open High Low
2024-05-09 00:00:00+00:00 61162.906250 61204.425781 61147.816406
2024-05-09 00:05:00+00:00 61224.207031 61224.207031 61186.906250
2024-05-09 00:10:00+00:00 61194.218750 61308.046875 61182.968750
2024-05-09 00:15:00+00:00 61247.027344 61266.984375 61221.957031
2024-05-09 00:20:00+00:00 61208.257812 61288.429688 61208.257812
... ... ... ...
2024-05-09 23:30:00+00:00 62953.804688 62975.207031 62940.707031
2024-05-09 23:35:00+00:00 62979.972656 62979.972656 62883.500000
2024-05-09 23:40:00+00:00 62895.757812 62899.800781 62886.890625
2024-05-09 23:45:00+00:00 62927.777344 62965.730469 62927.777344
2024-05-09 23:50:00+00:00 63046.648438 63147.433594 63046.648438

                                           Close     Adj Close      Volume  Dividends  \
         2024-05-09 00:00:00+00:00  61204.425781  61204.425781        -1.0        0.0   
         2024-05-09 00:05:00+00:00  61186.906250  61186.906250        -1.0        0.0   
         2024-05-09 00:10:00+00:00  61260.464844  61260.464844   2453504.0        0.0   
         2024-05-09 00:15:00+00:00  61221.957031  61221.957031   2572288.0        0.0   
         2024-05-09 00:20:00+00:00  61288.429688  61288.429688        -1.0        0.0   
         ...                                 ...           ...         ...        ...   
         2024-05-09 23:30:00+00:00  62975.207031  62975.207031        -1.0        0.0   
         2024-05-09 23:35:00+00:00  62883.500000  62883.500000        -1.0        0.0   
         2024-05-09 23:40:00+00:00  62892.656250  62892.656250        -1.0        0.0   
         2024-05-09 23:45:00+00:00  62965.730469  62965.730469        -1.0        0.0   
         2024-05-09 23:50:00+00:00  63147.433594  63147.433594  23889920.0        0.0   
         
                                    Stock Splits  Repaired?  
         2024-05-09 00:00:00+00:00           0.0      False  
         2024-05-09 00:05:00+00:00           0.0      False  
         2024-05-09 00:10:00+00:00           0.0      False  
         2024-05-09 00:15:00+00:00           0.0      False  
         2024-05-09 00:20:00+00:00           0.0      False  
         ...                                 ...        ...  
         2024-05-09 23:30:00+00:00           0.0      False  
         2024-05-09 23:35:00+00:00           0.0      False  
         2024-05-09 23:40:00+00:00           0.0      False  
         2024-05-09 23:45:00+00:00           0.0      False  
         2024-05-09 23:50:00+00:00           0.0      False  
         
         [287 rows x 9 columns]

DEBUG Fetching 2m prepost=False 2024-05-08->2024-05-11
DEBUG Entering history()
DEBUG BTC-USD: Yahoo GET parameters: {'period1': '2024-05-08 00:00:00+00:00', 'period2': '2024-05-11 00:00:00+00:00', 'interval': '2m', 'includePrePost': False, 'events': 'div,splits,capitalGains'}
DEBUG BTC-USD: yfinance received OHLC data: 2024-05-08 00:00:00 -> 2024-05-10 23:58:00
DEBUG BTC-USD: OHLC after cleaning: 2024-05-08 00:00:00+00:00 -> 2024-05-10 23:58:00+00:00
DEBUG BTC-USD: OHLC after combining events: 2024-05-08 00:00:00+00:00 -> 2024-05-10 23:58:00+00:00
DEBUG BTC-USD: checking OHLC for repairs ...
DEBUG Entering _fix_unit_mixups()
DEBUG Entering _fix_unit_switch()
DEBUG Entering _fix_prices_sudden_change()
DEBUG price-repair-split: appears_suspended=True, idx_latest_active=4 (2024-05-10)
INFO price-repair-split: No 100x errors detected
DEBUG Exiting _fix_prices_sudden_change()
DEBUG Exiting _fix_unit_switch()
DEBUG Entering _fix_unit_random_mixups()
INFO price-repair-100x: No sporadic 100x errors
DEBUG Exiting _fix_unit_random_mixups()
DEBUG Exiting _fix_unit_mixups()
DEBUG Entering _fix_bad_stock_split()
DEBUG Exiting _fix_bad_stock_split()
DEBUG Entering _fix_zeroes()
DEBUG Entering _reconstruct_intervals_batch()
DEBUG min_dt=2024-04-17 00:00:00+00:00 interval=2m sub_interval=1m
DEBUG grp_max_size = 5 days, 0:00:00
DEBUG Repair groups:
DEBUG - 2024-05-08 00:02:00+00:00 -> 2024-05-10 23:58:00+00:00
DEBUG df_block:
Open High Low
2024-05-08 00:00:00+00:00 62322.082031 62343.535156 62322.082031
2024-05-08 00:02:00+00:00 62367.093750 62412.796875 62367.093750
2024-05-08 00:04:00+00:00 62380.914062 62380.914062 62380.472656
2024-05-08 00:06:00+00:00 62374.738281 62374.738281 62374.738281
2024-05-08 00:08:00+00:00 62438.960938 62457.699219 62438.960938
... ... ... ...
2024-05-10 23:50:00+00:00 60899.343750 60899.343750 60898.917969
2024-05-10 23:52:00+00:00 60903.011719 60903.011719 60898.910156
2024-05-10 23:54:00+00:00 60881.300781 60881.300781 60881.300781
2024-05-10 23:56:00+00:00 60785.386719 60806.562500 60785.386719
2024-05-10 23:58:00+00:00 -1.000000 -1.000000 -1.000000

                                              Close     Adj Close      Volume  Dividends  \
            2024-05-08 00:00:00+00:00  62343.535156  62343.535156   5607424.0        0.0   
            2024-05-08 00:02:00+00:00  62412.796875  62412.796875        -1.0        0.0   
            2024-05-08 00:04:00+00:00  62380.472656  62380.472656  15757312.0        0.0   
            2024-05-08 00:06:00+00:00  62374.738281  62374.738281         0.0        0.0   
            2024-05-08 00:08:00+00:00  62457.699219  62457.699219  16173056.0        0.0   
            ...                                 ...           ...         ...        ...   
            2024-05-10 23:50:00+00:00  60898.917969  60898.917969        -1.0        0.0   
            2024-05-10 23:52:00+00:00  60898.910156  60898.910156        -1.0        0.0   
            2024-05-10 23:54:00+00:00  60881.300781  60881.300781         0.0        0.0   
            2024-05-10 23:56:00+00:00  60806.562500  60806.562500    673792.0        0.0   
            2024-05-10 23:58:00+00:00     -1.000000     -1.000000        -1.0        0.0   
            
                                       Stock Splits  Repaired?  
            2024-05-08 00:00:00+00:00           0.0      False  
            2024-05-08 00:02:00+00:00           0.0      False  
            2024-05-08 00:04:00+00:00           0.0      False  
            2024-05-08 00:06:00+00:00           0.0      False  
            2024-05-08 00:08:00+00:00           0.0      False  
            ...                                 ...        ...  
            2024-05-10 23:50:00+00:00           0.0      False  
            2024-05-10 23:52:00+00:00           0.0      False  
            2024-05-10 23:54:00+00:00           0.0      False  
            2024-05-10 23:56:00+00:00           0.0      False  
            2024-05-10 23:58:00+00:00           0.0      False  
            
            [2160 rows x 9 columns]

DEBUG Fetching 1m prepost=False 2024-05-07->2024-05-13
DEBUG Entering history()
DEBUG BTC-USD: Yahoo GET parameters: {'period1': '2024-05-07 00:00:00+00:00', 'period2': '2024-05-13 00:00:00+00:00', 'interval': '1m', 'includePrePost': False, 'events': 'div,splits,capitalGains'}
DEBUG BTC-USD: yfinance received OHLC data: 2024-05-07 00:00:00 -> 2024-05-12 23:58:00
DEBUG BTC-USD: OHLC after cleaning: 2024-05-07 00:00:00+00:00 -> 2024-05-12 23:58:00+00:00
DEBUG BTC-USD: OHLC after combining events: 2024-05-07 00:00:00+00:00 -> 2024-05-12 23:58:00+00:00
DEBUG BTC-USD: checking OHLC for repairs ...
DEBUG Entering _fix_unit_mixups()
DEBUG Entering _fix_unit_switch()
DEBUG Entering _fix_prices_sudden_change()
DEBUG price-repair-split: appears_suspended=False, idx_latest_active=1 (2024-05-12)
INFO price-repair-split: No 100x errors detected
DEBUG Exiting _fix_prices_sudden_change()
DEBUG Exiting _fix_unit_switch()
DEBUG Entering _fix_unit_random_mixups()
INFO price-repair-100x: No sporadic 100x errors
DEBUG Exiting _fix_unit_random_mixups()
DEBUG Exiting _fix_unit_mixups()
DEBUG Entering _fix_bad_stock_split()
DEBUG Exiting _fix_bad_stock_split()
DEBUG Entering _fix_zeroes()
DEBUG Entering _reconstruct_intervals_batch()
DEBUG Exiting _reconstruct_intervals_batch()
DEBUG Exiting _fix_zeroes()
DEBUG Entering _fix_missing_div_adjust()
DEBUG Exiting _fix_missing_div_adjust()
DEBUG BTC-USD: yfinance returning OHLC: 2024-05-07 00:00:00+00:00 -> 2024-05-12 23:58:00+00:00
DEBUG Exiting history()
DEBUG df_new:
Open Close Adj Close
2024-05-08 00:00:00+00:00 62322.082031 62343.535156 62343.535156
2024-05-08 00:02:00+00:00 62367.093750 62412.796875 62412.796875
2024-05-08 00:04:00+00:00 62380.914062 62380.472656 62380.472656
2024-05-08 00:06:00+00:00 62374.738281 62374.738281 62374.738281
2024-05-08 00:08:00+00:00 62438.960938 62457.699219 62457.699219
... ... ... ...
2024-05-10 23:50:00+00:00 60899.343750 60898.917969 60898.917969
2024-05-10 23:52:00+00:00 60903.011719 60898.910156 60898.910156
2024-05-10 23:54:00+00:00 60881.300781 60881.300781 60881.300781
2024-05-10 23:56:00+00:00 60785.386719 60806.562500 60806.562500
2024-05-10 23:58:00+00:00 NaN NaN NaN

                                                Low          High  Dividends    Volume  
            2024-05-08 00:00:00+00:00  62322.082031  62343.535156        0.0   6049792  
            2024-05-08 00:02:00+00:00  62367.093750  62412.796875        0.0         0  
            2024-05-08 00:04:00+00:00  62380.472656  62380.914062        0.0  15757312  
            2024-05-08 00:06:00+00:00  62374.738281  62374.738281        0.0         0  
            2024-05-08 00:08:00+00:00  62438.960938  62457.699219        0.0  16173056  
            ...                                 ...           ...        ...       ...  
            2024-05-10 23:50:00+00:00  60898.917969  60899.343750        0.0         0  
            2024-05-10 23:52:00+00:00  60898.910156  60903.011719        0.0         0  
            2024-05-10 23:54:00+00:00  60881.300781  60881.300781        0.0         0  
            2024-05-10 23:56:00+00:00  60785.386719  60806.562500        0.0    673792  
            2024-05-10 23:58:00+00:00           NaN           NaN        0.0         0  
            
            [2160 rows x 7 columns]

DEBUG Price calibration ratio (raw) = 1.000000
DEBUG Exiting _reconstruct_intervals_batch()
INFO price-repair-missing: BTC-USD: fixed 616/616 value=0 errors in 2m price data
DEBUG Exiting _fix_zeroes()
DEBUG Entering _fix_missing_div_adjust()
DEBUG Exiting _fix_missing_div_adjust()
DEBUG BTC-USD: yfinance returning OHLC: 2024-05-08 00:00:00+00:00 -> 2024-05-10 23:58:00+00:00
DEBUG Exiting history()
DEBUG df_new:
Open Close Adj Close
2024-05-09 00:00:00+00:00 61162.906250 61186.906250 61186.906250
2024-05-09 00:10:00+00:00 61194.218750 61221.957031 61221.957031
2024-05-09 00:20:00+00:00 61208.257812 61354.125000 61354.125000
2024-05-09 00:30:00+00:00 61343.410156 61353.480469 61353.480469
2024-05-09 00:40:00+00:00 61357.382812 61403.101562 61403.101562
... ... ... ...
2024-05-09 23:10:00+00:00 63128.957031 62976.238281 62976.238281
2024-05-09 23:20:00+00:00 62987.125000 62955.250000 62955.250000
2024-05-09 23:30:00+00:00 62953.804688 62883.500000 62883.500000
2024-05-09 23:40:00+00:00 62895.757812 62965.730469 62965.730469
2024-05-09 23:50:00+00:00 63046.648438 63098.777344 63098.777344

                                             Low          High  Dividends    Volume  
         2024-05-09 00:00:00+00:00  61147.816406  61224.207031        0.0   4972544  
         2024-05-09 00:10:00+00:00  61182.968750  61308.046875        0.0  17688576  
         2024-05-09 00:20:00+00:00  61208.257812  61354.125000        0.0   4491264  
         2024-05-09 00:30:00+00:00  61304.511719  61361.765625        0.0   4648960  
         2024-05-09 00:40:00+00:00  61357.382812  61435.660156        0.0         0  
         ...                                 ...           ...        ...       ...  
         2024-05-09 23:10:00+00:00  62976.238281  63128.957031        0.0   8067072  
         2024-05-09 23:20:00+00:00  62902.449219  62987.125000        0.0  33947648  
         2024-05-09 23:30:00+00:00  62883.500000  62979.972656        0.0    833536  
         2024-05-09 23:40:00+00:00  62886.890625  62965.730469        0.0    301056  
         2024-05-09 23:50:00+00:00  63046.648438  63147.433594        0.0  38248448  
         
         [144 rows x 7 columns]

DEBUG Price calibration ratio (raw) = 0.999958
DEBUG Yahoo didn't return finer-grain data for these intervals: [Timestamp('2024-05-09 00:05:00+0000', tz='UTC'), Timestamp('2024-05-09 00:25:00+0000', tz='UTC'), Timestamp('2024-05-09 00:35:00+0000', tz='UTC'), Timestamp('2024-05-09 00:45:00+0000', tz='UTC'), Timestamp('2024-05-09 00:55:00+0000', tz='UTC'), Timestamp('2024-05-09 01:05:00+0000', tz='UTC'), Timestamp('2024-05-09 01:15:00+0000', tz='UTC'), Timestamp('2024-05-09 01:25:00+0000', tz='UTC'), Timestamp('2024-05-09 01:35:00+0000', tz='UTC'), Timestamp('2024-05-09 01:45:00+0000', tz='UTC'), Timestamp('2024-05-09 02:05:00+0000', tz='UTC'), Timestamp('2024-05-09 02:15:00+0000', tz='UTC'), Timestamp('2024-05-09 02:25:00+0000', tz='UTC'), Timestamp('2024-05-09 02:55:00+0000', tz='UTC'), Timestamp('2024-05-09 03:15:00+0000', tz='UTC'), Timestamp('2024-05-09 03:45:00+0000', tz='UTC'), Timestamp('2024-05-09 04:05:00+0000', tz='UTC'), Timestamp('2024-05-09 04:25:00+0000', tz='UTC'), Timestamp('2024-05-09 04:35:00+0000', tz='UTC'), Timestamp('2024-05-09 04:45:00+0000', tz='UTC'), Timestamp('2024-05-09 04:55:00+0000', tz='UTC'), Timestamp('2024-05-09 05:05:00+0000', tz='UTC'), Timestamp('2024-05-09 05:15:00+0000', tz='UTC'), Timestamp('2024-05-09 05:55:00+0000', tz='UTC'), Timestamp('2024-05-09 06:05:00+0000', tz='UTC'), Timestamp('2024-05-09 06:25:00+0000', tz='UTC'), Timestamp('2024-05-09 06:35:00+0000', tz='UTC'), Timestamp('2024-05-09 06:45:00+0000', tz='UTC'), Timestamp('2024-05-09 06:55:00+0000', tz='UTC'), Timestamp('2024-05-09 07:05:00+0000', tz='UTC'), Timestamp('2024-05-09 07:15:00+0000', tz='UTC'), Timestamp('2024-05-09 07:25:00+0000', tz='UTC'), Timestamp('2024-05-09 07:35:00+0000', tz='UTC'), Timestamp('2024-05-09 07:55:00+0000', tz='UTC'), Timestamp('2024-05-09 08:05:00+0000', tz='UTC'), Timestamp('2024-05-09 08:15:00+0000', tz='UTC'), Timestamp('2024-05-09 08:35:00+0000', tz='UTC'), Timestamp('2024-05-09 08:45:00+0000', tz='UTC'), Timestamp('2024-05-09 08:55:00+0000', tz='UTC'), Timestamp('2024-05-09 09:15:00+0000', tz='UTC'), Timestamp('2024-05-09 09:25:00+0000', tz='UTC'), Timestamp('2024-05-09 09:35:00+0000', tz='UTC'), Timestamp('2024-05-09 09:55:00+0000', tz='UTC'), Timestamp('2024-05-09 10:15:00+0000', tz='UTC'), Timestamp('2024-05-09 10:25:00+0000', tz='UTC'), Timestamp('2024-05-09 10:45:00+0000', tz='UTC'), Timestamp('2024-05-09 12:25:00+0000', tz='UTC'), Timestamp('2024-05-09 13:05:00+0000', tz='UTC'), Timestamp('2024-05-09 13:15:00+0000', tz='UTC'), Timestamp('2024-05-09 13:25:00+0000', tz='UTC'), Timestamp('2024-05-09 13:35:00+0000', tz='UTC'), Timestamp('2024-05-09 13:55:00+0000', tz='UTC'), Timestamp('2024-05-09 14:25:00+0000', tz='UTC'), Timestamp('2024-05-09 14:45:00+0000', tz='UTC'), Timestamp('2024-05-09 14:55:00+0000', tz='UTC'), Timestamp('2024-05-09 15:15:00+0000', tz='UTC'), Timestamp('2024-05-09 17:45:00+0000', tz='UTC'), Timestamp('2024-05-09 18:15:00+0000', tz='UTC'), Timestamp('2024-05-09 18:25:00+0000', tz='UTC'), Timestamp('2024-05-09 18:35:00+0000', tz='UTC'), Timestamp('2024-05-09 18:45:00+0000', tz='UTC'), Timestamp('2024-05-09 19:05:00+0000', tz='UTC'), Timestamp('2024-05-09 19:15:00+0000', tz='UTC'), Timestamp('2024-05-09 19:25:00+0000', tz='UTC'), Timestamp('2024-05-09 19:45:00+0000', tz='UTC'), Timestamp('2024-05-09 20:15:00+0000', tz='UTC'), Timestamp('2024-05-09 20:25:00+0000', tz='UTC'), Timestamp('2024-05-09 20:35:00+0000', tz='UTC'), Timestamp('2024-05-09 20:45:00+0000', tz='UTC'), Timestamp('2024-05-09 20:55:00+0000', tz='UTC'), Timestamp('2024-05-09 21:25:00+0000', tz='UTC'), Timestamp('2024-05-09 21:35:00+0000', tz='UTC'), Timestamp('2024-05-09 22:15:00+0000', tz='UTC'), Timestamp('2024-05-09 22:25:00+0000', tz='UTC'), Timestamp('2024-05-09 23:05:00+0000', tz='UTC'), Timestamp('2024-05-09 23:15:00+0000', tz='UTC'), Timestamp('2024-05-09 23:25:00+0000', tz='UTC'), Timestamp('2024-05-09 23:35:00+0000', tz='UTC'), Timestamp('2024-05-09 23:45:00+0000', tz='UTC')]
DEBUG Exiting _reconstruct_intervals_batch()
INFO price-repair-missing: BTC-USD: fixed 86/165 value=0 errors in 5m price data
DEBUG Exiting _fix_zeroes()
DEBUG Entering _fix_missing_div_adjust()
DEBUG Exiting _fix_missing_div_adjust()
DEBUG BTC-USD: yfinance returning OHLC: 2024-05-09 00:00:00+00:00 -> 2024-05-09 23:55:00+00:00
DEBUG Exiting history()
DEBUG Exiting history()

Bad data proof

imagen

yfinance version

0.2.38

Python version

3.12.3

Operating system

Windows 11
@jvallsgm
Copy link
Author

I did not explain myself well when I referred to negative values. Ignore this part. The problem is the many zeros that appear, and they are incorrect.
Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jvallsgm